Remove trailing periods to avoid mangled commands

Frontend was still passing the project's URL to the backend even though
the backend had been refactored to use the project's ID. This is fixed.

Deleting the repo's files should also work now, but I do think I want to
support tracking multiple instances of the same project. This means
we'll need to check for other instances relying on the repo before
deleting it.

We need end-to-end tests 😩😩😩

.files/2024-02-24.png.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
+
+SPDX-License-Identifier: CC0-1.0

.gitignore

@@ -5,7 +5,7 @@
 /willow
 /*.csv
 /data/
-/*.sqlite
+/*.sqlite*
 /config.toml
 
 /.idea/

README.md

@@ -12,74 +12,134 @@ SPDX-License-Identifier: CC0-1.0
 
 _Forge-agnostic software release tracker_
 
-![screenshot of willow's current web UI](.files/2023-10-29.png)
+![screenshot of willow's current web UI](.files/2024-02-24.png)
 
-_This UI is a minimal proof-of-concept, it's going to change drastically in the
-near future._
+_This UI is Amolith's attempt at a balance between simple, pleasant, and
+functional. Amolith is not a UX professional and would **very** much welcome
+input from someone more knowledgeable!_
 
 ## What is it?
 
-_If you'd rather watch a video, I gave a [lightning talk on Willow] at the 2023
-Ubuntu Summit._
+_If you'd rather watch a short video, Amolith gave a 5-minute [lightning talk on
+Willow] at the 2023 Ubuntu Summit._
 
 [lightning talk on Willow]: https://youtu.be/XIGxKyekvBQ?t=29900
 
-Willow tracks software releases across arbitrary forge platforms by trying to
-support one of the very few things they all have in common: the VCS. At the
-moment, git is the _only_ supported VCS, but I would be interested in adding
-Pijul, Fossil, Mercurial, etc. You can also track releases using RSS feeds.
+Willow helps developers, sysadmins, and homelabbers keep up with software
+releases across arbitrary forge platforms, including full-featured forges like
+GitHub, GitLab, or [Forgejo] as well as more minimal options like [cgit] or
+[stagit].
 
-Willow exists because decentralisation can be annoying. One piece of software
-can be found on GitHub, another piece on GitLab, one on Bitbucket, a fourth on
-SourceHut, and a fifth on the developer's self-hosted Forgejo instance. Forgejo
-and GitHub have RSS feeds that only notify you of releases. GitLab doesn't
-support RSS feeds for anything, just an API you can poke. Some software updates
-might be on the developers' personal blog. Sometimes there are CVEs for specific
-software and they get published somewhere completely different before they're
-fixed in a release.
+[Forgejo]: https://forgejo.org/
+[cgit]: https://git.zx2c4.com/cgit/
+[stagit]: https://codemadness.org/stagit.html
 
-I want to bring all that scattered information under one roof so a developer or
-sysadmin can pop open willow's web UI and immediately see what needs updating
-where. I've recorded some of my other ideas and plans in [my wiki].
+It exists because decentralisation, as wonderful as it is, does have some pain
+points. One piece of software is on GitHub, another piece is on GitLab, one on
+Bitbucket, a fourth on [SourceHut], a fifth on the developer's self-hosted
+Forgejo instance.
 
-[my wiki]: https://wiki.secluded.site/hypha/willow
+[SourceHut]: https://sourcehut.org/
+
+The capabilities of each platform can also differ, further complicating the
+space. For example, Forgejo and GitHub have APIs and RSS release feeds,
+SourceHut has an API and RSS feeds that notify you of _all_ activity in the
+repo, GitLab only has an API, and there's no standard for discovering the
+capabilities of arbitrary git frontends like [legit].
+
+[legit]: https://github.com/icyphox/legit
+
+And _then_ you have different pieces of information in different places; some
+developers might publish release announcements on their personal blog and some
+projects might release security advisories on an external platform prior to
+publishing a release.
+
+All this important info is scattered all over the internet. Willow brings some
+order to that chaos by supporting both RSS and one of the _very_ few things all
+the forges and frontends have in common: their **V**ersion **C**ontrol
+**S**ystem. At the moment, [Git] is the _only_ supported VCS, but we're
+definitely interested in adding support for [Pijul], [Fossil], [Mercurial], and
+potentially others.
+
+[Git]: https://git-scm.com/
+[Pijul]: https://pijul.org/
+[Fossil]: https://www.fossil-scm.org/
+[Mercurial]: https://www.mercurial-scm.org/
+
+Amolith (the creator) has recorded some of his other ideas, thoughts, and plans
+in [his wiki].
+
+[his wiki]: https://wiki.secluded.site/hypha/willow
 
 ## Installation and use
 
-_**Note:** prebuilt binaries will be available after I release [v0.0.1]_
+**Disclaimers:** 
+1. Prebuilt binaries will be available with the [v0.0.1] release, greatly
+   simplifying installation.
+2. We consider the project _alpha-quality_. There will be bugs.
+3. Amolith has tried to make the web UI accessible, but is unsure of its current
+   usability.
+4. The app is not localised yet and English is the only available language.
+5. Help with any/all of the above is most welcome!
 
 [v0.0.1]: https://todo.sr.ht/~amolith/willow?search=status%3Aopen%20label%3A%22v0.0.1%22
+[communication platforms]: #contributing
+
+### Installation
+
+This assumes Willow will run on an always-on server, like a VPS.
+
+* Clone the repo with `git clone https://git.sr.ht/~amolith/willow`
+* Enter the repo's folder with `cd willow`
+* Build the binary with `CGO_ENABLED=0 go build -ldflags="-s -w" -o willow
+  ./cmd`
+* Transfer the binary to the server however you like
+* Execute the binary with `./willow`
+* Edit the config with `vim config.toml`
+* Daemonise Willow using systemd or OpenRC or whatever you prefer
+* Reverse-proxy the web UI (defaults to `localhost:1313`) with Caddy or NGINX or
+  whatever you prefer
+
+### Use
 
-* Clone the repo
-* Build the binary with `CGO_ENABLED=0 go build -ldflags="-s -w" -o willow ./cmd`
-* Upload it to a remote server
-* Execute the binary
-* Edit the `config.toml`
 * Create a user with `./willow -a <username>`
-* Execute the binary again
-* Reverse proxy `http://localhost:1313`
-* Open the web UI
+* Open the web UI (defaults to `localhost:1313`, but [installation] had you put
+  a proxy in front)
 * Click `Track new project`
-* Fill out the form
-* Indicate which version you're currently on
-* That's it!
+* Fill out the form and press `Next`
+* Indicate which version you're currently on and press `Track releases`
+* You're now tracking that project's releases!
 
-Note that I still consider the project to be in _alpha_ state. There will be
-bugs; please help fix them!
+[installation]: #installation
+
+If you no longer use that project, click the `Delete?` link to remove it, and,
+if applicable, Willow's copy of its repo.
+
+If you're no longer running the version Willow says you've selected, click the
+`Modify?` link to select a different version.
+
+If there are projects where your selected version does _not_ match what Willow
+thinks is latest, they'll show up at the top under the **Outdated projects**
+heading and have a link at the bottom of the card to `View release notes`.
+Clicking that link populates the right column with those release notes.
+
+If there are projects where your selected version _does_ match what Willow
+thinks is latest, they'll show up at the bottom under the **Up-to-date
+projects** heading.
 
 ## Contributing
 
 Contributions are very much welcome! Please take a look at the [ticket
 tracker][todo] and see if there's anything you're interested in working on. If
 there's specific functionality you'd like to see implemented and it's not
-mentioned in the ticket tracker, please send a description to the [mailing
-list][email] so we can discuss its inclusion. If I don't feel like it fits with
+mentioned in the ticket tracker, please describe it through one of the platforms
+below so we can discuss its inclusion. If we don't feel like it fits with
 Willow's goals, you're encouraged to fork the project and make whatever changes
 you like!
 
-Questions, comments, and patches can always be sent to the [mailing
-list][email], but I'm also in the [IRC channel][irc]/[XMPP room][xmpp] pretty
-much 24/7. I might not see messages right away, so please stick around.
+Questions, comments, and patches can always go to the [mailing list][email], but
+there's also an [IRC channel][irc] and an [XMPP MUC][xmpp] for real-time
+interactions.
 
 - Email: [~amolith/willow@lists.sr.ht][email]
 - IRC: [irc.libera.chat/#willow][irc]

db/project.go

@@ -22,9 +22,9 @@ func DeleteProject(db *sql.DB, mu *sync.Mutex, id string) error {
 }
 
 // GetProject returns a project from the database
-func GetProject(db *sql.DB, url string) (map[string]string, error) {
-	var id, name, forge, version string
-	err := db.QueryRow("SELECT id, name, forge, version FROM projects WHERE url = ?", url).Scan(&id, &name, &forge, &version)
+func GetProject(db *sql.DB, id string) (map[string]string, error) {
+	var name, forge, url, version string
+	err := db.QueryRow("SELECT name, forge, url, version FROM projects WHERE id = ?", id).Scan(&name, &forge, &url, &version)
 	if err != nil {
 		return nil, err
 	}

db/release.go

@@ -9,7 +9,7 @@ import (
 	"sync"
 )
 
-// UpsertRelease adds or updates a release for a project with a given URL in the
+// UpsertRelease adds or updates a release for a project with a given ID in the
 // database
 func UpsertRelease(db *sql.DB, mu *sync.Mutex, id, projectID, url, tag, content, date string) error {
 	mu.Lock()
@@ -48,11 +48,12 @@ func GetReleases(db *sql.DB, projectID string) ([]map[string]string, error) {
 			return nil, err
 		}
 		releases = append(releases, map[string]string{
-			"id":      id,
-			"url":     url,
-			"tag":     tag,
-			"content": content,
-			"date":    date,
+			"id":         id,
+			"project_id": projectID,
+			"url":        url,
+			"tag":        tag,
+			"content":    content,
+			"date":       date,
 		})
 	}
 	return releases, nil

git/git.go

@@ -7,7 +7,6 @@ package git
 import (
 	"errors"
 	"fmt"
-	"io"
 	"net/url"
 	"os"
 	"strings"
@@ -149,42 +148,23 @@ func RemoveRepo(url string) (err error) {
 		return err
 	}
 
-	// TODO: Check whether the two parent directories are empty and remove them if
-	// so
-	for i := 0; i < 2; i++ {
-		path = strings.TrimSuffix(path, "/")
+	path = path[:strings.LastIndex(path, "/")]
+	dirs := strings.Split(path, "/")
+
+	for range dirs {
 		if path == "data" {
 			break
 		}
-		empty, err := dirEmpty(path)
+		err = os.Remove(path)
 		if err != nil {
-			return err
-		}
-		if empty {
-			err = os.Remove(path)
-			if err != nil {
-				return err
-			}
+			// This folder likely has data, so might as well save some time by
+			// not checking the parents we can't delete anyway.
+			break
 		}
 		path = path[:strings.LastIndex(path, "/")]
 	}
 
-	return err
-}
-
-// dirEmpty checks if a directory is empty.
-func dirEmpty(name string) (empty bool, err error) {
-	f, err := os.Open(name)
-	if err != nil {
-		return false, err
-	}
-	defer f.Close()
-
-	_, err = f.Readdirnames(1)
-	if err == io.EOF {
-		return true, nil
-	}
-	return false, err
+	return nil
 }
 
 // stringifyRepo accepts a repository URI string and the corresponding local

project/project.go

@@ -7,6 +7,7 @@ package project
 import (
 	"crypto/sha256"
 	"database/sql"
+	"errors"
 	"fmt"
 	"log"
 	"sort"
@@ -41,22 +42,33 @@ type Release struct {
 
 // GetReleases returns a list of all releases for a project from the database
 func GetReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
+	proj.ID = GenProjectID(proj.URL, proj.Name, proj.Forge)
+
 	ret, err := db.GetReleases(dbConn, proj.ID)
 	if err != nil {
 		return proj, err
 	}
 
 	if len(ret) == 0 {
-		return fetchReleases(dbConn, mu, proj)
+		proj, err = fetchReleases(dbConn, mu, proj)
+		if err != nil {
+			return proj, err
+		}
+		err = upsertReleases(dbConn, mu, proj.ID, proj.Releases)
+		if err != nil {
+			return proj, err
+		}
+		return proj, nil
 	}
 
 	for _, row := range ret {
 		proj.Releases = append(proj.Releases, Release{
-			ID:      row["id"],
-			Tag:     row["tag"],
-			Content: row["content"],
-			URL:     row["release_url"],
-			Date:    time.Time{},
+			ID:        row["id"],
+			ProjectID: proj.ID,
+			Tag:       row["tag"],
+			Content:   row["content"],
+			URL:       row["release_url"],
+			Date:      time.Time{},
 		})
 	}
 	proj.Releases = SortReleases(proj.Releases)
@@ -81,7 +93,7 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertRelease(dbConn, mu, p.URL, p.Releases)
+			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -100,7 +112,7 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertRelease(dbConn, mu, p.URL, p.Releases)
+			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -118,12 +130,18 @@ func SortReleases(releases []Release) []Release {
 	return releases
 }
 
-// upsertRelease updates or inserts a release in the database
-func upsertRelease(dbConn *sql.DB, mu *sync.Mutex, url string, releases []Release) error {
+func SortProjects(projects []Project) []Project {
+	sort.Slice(projects, func(i, j int) bool {
+		return strings.ToLower(projects[i].Name) < strings.ToLower(projects[j].Name)
+	})
+	return projects
+}
+
+// upsertReleases updates or inserts a release in the database
+func upsertReleases(dbConn *sql.DB, mu *sync.Mutex, projID string, releases []Release) error {
 	for _, release := range releases {
 		date := release.Date.Format("2006-01-02 15:04:05")
-		id := GenReleaseID(url, release.URL, release.Tag)
-		err := db.UpsertRelease(dbConn, mu, id, url, release.URL, release.Tag, release.Content, date)
+		err := db.UpsertRelease(dbConn, mu, release.ID, projID, release.URL, release.Tag, release.Content, date)
 		if err != nil {
 			log.Printf("Error upserting release: %v", err)
 			return err
@@ -154,12 +172,19 @@ func Track(dbConn *sql.DB, mu *sync.Mutex, manualRefresh *chan struct{}, name, u
 }
 
 func Untrack(dbConn *sql.DB, mu *sync.Mutex, id string) {
-	err := db.DeleteProject(dbConn, mu, id)
+	proj, err := db.GetProject(dbConn, id)
+	if err != nil {
+		fmt.Println("Error getting project:", err)
+	}
+
+	err = db.DeleteProject(dbConn, mu, proj["id"])
 	if err != nil {
 		fmt.Println("Error deleting project:", err)
 	}
 
-	err = git.RemoveRepo(id)
+	// TODO: before removing, check whether other tracked projects use the same
+	// repo
+	err = git.RemoveRepo(proj["url"])
 	if err != nil {
 		log.Println(err)
 	}
@@ -185,7 +210,7 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 			return strings.ToLower(projectsList[i].Name) < strings.ToLower(projectsList[j].Name)
 		})
 		for i := range projectsList {
-			err = upsertRelease(dbConn, mu, projectsList[i].URL, projectsList[i].Releases)
+			err = upsertReleases(dbConn, mu, projectsList[i].ID, projectsList[i].Releases)
 			if err != nil {
 				fmt.Println("Error upserting release:", err)
 				continue
@@ -212,24 +237,26 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 }
 
 // GetProject returns a project from the database
-func GetProject(dbConn *sql.DB, id string) (Project, error) {
-	projectDB, err := db.GetProject(dbConn, id)
-	if err != nil {
-		return Project{}, err
+func GetProject(dbConn *sql.DB, proj Project) (Project, error) {
+	projectDB, err := db.GetProject(dbConn, proj.ID)
+	if err != nil && errors.Is(err, sql.ErrNoRows) {
+		return proj, nil
+	} else if err != nil {
+		return proj, err
 	}
 	p := Project{
-		ID:      projectDB["id"],
-		URL:     projectDB["url"],
-		Name:    projectDB["name"],
-		Forge:   projectDB["forge"],
+		ID:      proj.ID,
+		URL:     proj.URL,
+		Name:    proj.Name,
+		Forge:   proj.Forge,
 		Running: projectDB["version"],
 	}
 	return p, err
 }
 
 // GetProjectWithReleases returns a single project from the database along with its releases
-func GetProjectWithReleases(dbConn *sql.DB, mu *sync.Mutex, id string) (Project, error) {
-	project, err := GetProject(dbConn, id)
+func GetProjectWithReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
+	project, err := GetProject(dbConn, proj)
 	if err != nil {
 		return Project{}, err
 	}
@@ -255,7 +282,7 @@ func GetProjects(dbConn *sql.DB) ([]Project, error) {
 		}
 	}
 
-	return projects, nil
+	return SortProjects(projects), nil
 }
 
 // GetProjectsWithReleases returns a list of all projects and all their releases
@@ -274,5 +301,5 @@ func GetProjectsWithReleases(dbConn *sql.DB, mu *sync.Mutex) ([]Project, error)
 		projects[i].Releases = SortReleases(projects[i].Releases)
 	}
 
-	return projects, nil
+	return SortProjects(projects), nil
 }

ws/static/home.html

@@ -34,7 +34,7 @@
                 {{- range . -}}
                 {{- if ne .Running (index .Releases 0).Tag -}}
                 <div id="{{ .ID }}" class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h3>
+                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
                     <p>You've selected {{ .Running }}. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
                     <p>Latest: <a href="{{ (index .Releases 0).URL }}">{{ (index .Releases 0).Tag }}</a></p>
                     <p><a href="#{{ (index .Releases 0).ID }}">View release notes</a></p>
@@ -52,7 +52,7 @@
                 {{- range . -}}
                 {{- if eq .Running (index .Releases 0).Tag -}}
                 <div class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h3>
+                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
                     <p>You've selected <a href="#{{ (index .Releases 0).ID }}">{{ .Running }}</a>. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
                 </div>
                 {{- end -}}

ws/static/select-release.html

@@ -44,7 +44,7 @@
             <input type="hidden" name="name" value="{{ .Name }}">
             <input type="hidden" name="forge" value="{{ .Forge }}">
             <input type="hidden" name="id" value="{{ .ID }}">
-            <input class="button" type="submit" formaction="/new" value="Track future releases">
+            <input class="button" type="submit" formaction="/new" value="Track releases">
         </form>
         <!-- Append these if they ever start limiting RSS entries: `(eq $forge "gitea") (eq $forge "forgejo")` -->
         {{- if or (eq $forge "github") -}}

ws/static/styles.css

@@ -128,7 +128,7 @@ header > h1 > span {
     }
 
     .card {
-        border: 2px solid #ccc;
+        border: 2px solid #424242;
         background: #1c1c1c;
     }
 

ws/ws.go

@@ -80,7 +80,34 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 
-			proj, err := project.GetProject(h.DbConn, submittedURL)
+			forge := bmStrict.Sanitize(params.Get("forge"))
+			if forge == "" {
+				w.WriteHeader(http.StatusBadRequest)
+				_, err := w.Write([]byte("No forge provided"))
+				if err != nil {
+					fmt.Println(err)
+				}
+				return
+			}
+
+			name := bmStrict.Sanitize(params.Get("name"))
+			if name == "" {
+				w.WriteHeader(http.StatusBadRequest)
+				_, err := w.Write([]byte("No name provided"))
+				if err != nil {
+					fmt.Println(err)
+				}
+				return
+			}
+
+			proj := project.Project{
+				ID:    project.GenProjectID(submittedURL, name, forge),
+				URL:   submittedURL,
+				Name:  name,
+				Forge: forge,
+			}
+
+			proj, err := project.GetProject(h.DbConn, proj)
 			if err != nil && err != sql.ErrNoRows {
 				w.WriteHeader(http.StatusBadRequest)
 				_, err := w.Write([]byte(fmt.Sprintf("Error getting project: %s", err)))
@@ -90,38 +117,6 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 
-			if proj.Running == "" {
-				forge := bmStrict.Sanitize(params.Get("forge"))
-				if forge == "" {
-					w.WriteHeader(http.StatusBadRequest)
-					_, err := w.Write([]byte("No forge provided"))
-					if err != nil {
-						fmt.Println(err)
-					}
-					return
-				}
-
-				name := bmStrict.Sanitize(params.Get("name"))
-				if name == "" {
-					w.WriteHeader(http.StatusBadRequest)
-					_, err := w.Write([]byte("No name provided"))
-					if err != nil {
-						fmt.Println(err)
-					}
-					return
-				}
-
-				proj = project.Project{
-					URL:   submittedURL,
-					Name:  name,
-					Forge: forge,
-				}
-
-				proj.URL = strings.TrimSuffix(proj.URL, ".git")
-
-			}
-
-			proj.ID = project.GenProjectID(proj.URL, proj.Name, proj.Forge)
 			proj, err = project.GetReleases(h.DbConn, h.Mu, proj)
 			if err != nil {
 				w.WriteHeader(http.StatusBadRequest)