Pass messages from backend to frontend

.files/2024-02-24.png.license

@@ -1,3 +0,0 @@
-SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
-
-SPDX-License-Identifier: CC0-1.0

.gitignore

@@ -5,7 +5,7 @@
 /willow
 /*.csv
 /data/
-/*.sqlite*
+/*.sqlite
 /config.toml
 
 /.idea/

README.md

@@ -12,134 +12,74 @@ SPDX-License-Identifier: CC0-1.0
 
 _Forge-agnostic software release tracker_
 
-![screenshot of willow's current web UI](.files/2024-02-24.png)
+![screenshot of willow's current web UI](.files/2023-10-29.png)
 
-_This UI is Amolith's attempt at a balance between simple, pleasant, and
-functional. Amolith is not a UX professional and would **very** much welcome
-input from someone more knowledgeable!_
+_This UI is a minimal proof-of-concept, it's going to change drastically in the
+near future._
 
 ## What is it?
 
-_If you'd rather watch a short video, Amolith gave a 5-minute [lightning talk on
-Willow] at the 2023 Ubuntu Summit._
+_If you'd rather watch a video, I gave a [lightning talk on Willow] at the 2023
+Ubuntu Summit._
 
 [lightning talk on Willow]: https://youtu.be/XIGxKyekvBQ?t=29900
 
-Willow helps developers, sysadmins, and homelabbers keep up with software
-releases across arbitrary forge platforms, including full-featured forges like
-GitHub, GitLab, or [Forgejo] as well as more minimal options like [cgit] or
-[stagit].
+Willow tracks software releases across arbitrary forge platforms by trying to
+support one of the very few things they all have in common: the VCS. At the
+moment, git is the _only_ supported VCS, but I would be interested in adding
+Pijul, Fossil, Mercurial, etc. You can also track releases using RSS feeds.
 
-[Forgejo]: https://forgejo.org/
-[cgit]: https://git.zx2c4.com/cgit/
-[stagit]: https://codemadness.org/stagit.html
+Willow exists because decentralisation can be annoying. One piece of software
+can be found on GitHub, another piece on GitLab, one on Bitbucket, a fourth on
+SourceHut, and a fifth on the developer's self-hosted Forgejo instance. Forgejo
+and GitHub have RSS feeds that only notify you of releases. GitLab doesn't
+support RSS feeds for anything, just an API you can poke. Some software updates
+might be on the developers' personal blog. Sometimes there are CVEs for specific
+software and they get published somewhere completely different before they're
+fixed in a release.
 
-It exists because decentralisation, as wonderful as it is, does have some pain
-points. One piece of software is on GitHub, another piece is on GitLab, one on
-Bitbucket, a fourth on [SourceHut], a fifth on the developer's self-hosted
-Forgejo instance.
+I want to bring all that scattered information under one roof so a developer or
+sysadmin can pop open willow's web UI and immediately see what needs updating
+where. I've recorded some of my other ideas and plans in [my wiki].
 
-[SourceHut]: https://sourcehut.org/
-
-The capabilities of each platform can also differ, further complicating the
-space. For example, Forgejo and GitHub have APIs and RSS release feeds,
-SourceHut has an API and RSS feeds that notify you of _all_ activity in the
-repo, GitLab only has an API, and there's no standard for discovering the
-capabilities of arbitrary git frontends like [legit].
-
-[legit]: https://github.com/icyphox/legit
-
-And _then_ you have different pieces of information in different places; some
-developers might publish release announcements on their personal blog and some
-projects might release security advisories on an external platform prior to
-publishing a release.
-
-All this important info is scattered all over the internet. Willow brings some
-order to that chaos by supporting both RSS and one of the _very_ few things all
-the forges and frontends have in common: their **V**ersion **C**ontrol
-**S**ystem. At the moment, [Git] is the _only_ supported VCS, but we're
-definitely interested in adding support for [Pijul], [Fossil], [Mercurial], and
-potentially others.
-
-[Git]: https://git-scm.com/
-[Pijul]: https://pijul.org/
-[Fossil]: https://www.fossil-scm.org/
-[Mercurial]: https://www.mercurial-scm.org/
-
-Amolith (the creator) has recorded some of his other ideas, thoughts, and plans
-in [his wiki].
-
-[his wiki]: https://wiki.secluded.site/hypha/willow
+[my wiki]: https://wiki.secluded.site/hypha/willow
 
 ## Installation and use
 
-**Disclaimers:** 
-1. Prebuilt binaries will be available with the [v0.0.1] release, greatly
-   simplifying installation.
-2. We consider the project _alpha-quality_. There will be bugs.
-3. Amolith has tried to make the web UI accessible, but is unsure of its current
-   usability.
-4. The app is not localised yet and English is the only available language.
-5. Help with any/all of the above is most welcome!
+_**Note:** prebuilt binaries will be available after I release [v0.0.1]_
 
 [v0.0.1]: https://todo.sr.ht/~amolith/willow?search=status%3Aopen%20label%3A%22v0.0.1%22
-[communication platforms]: #contributing
-
-### Installation
-
-This assumes Willow will run on an always-on server, like a VPS.
-
-* Clone the repo with `git clone https://git.sr.ht/~amolith/willow`
-* Enter the repo's folder with `cd willow`
-* Build the binary with `CGO_ENABLED=0 go build -ldflags="-s -w" -o willow
-  ./cmd`
-* Transfer the binary to the server however you like
-* Execute the binary with `./willow`
-* Edit the config with `vim config.toml`
-* Daemonise Willow using systemd or OpenRC or whatever you prefer
-* Reverse-proxy the web UI (defaults to `localhost:1313`) with Caddy or NGINX or
-  whatever you prefer
-
-### Use
 
+* Clone the repo
+* Build the binary with `CGO_ENABLED=0 go build -ldflags="-s -w" -o willow ./cmd`
+* Upload it to a remote server
+* Execute the binary
+* Edit the `config.toml`
 * Create a user with `./willow -a <username>`
-* Open the web UI (defaults to `localhost:1313`, but [installation] had you put
-  a proxy in front)
+* Execute the binary again
+* Reverse proxy `http://localhost:1313`
+* Open the web UI
 * Click `Track new project`
-* Fill out the form and press `Next`
-* Indicate which version you're currently on and press `Track releases`
-* You're now tracking that project's releases!
+* Fill out the form
+* Indicate which version you're currently on
+* That's it!
 
-[installation]: #installation
-
-If you no longer use that project, click the `Delete?` link to remove it, and,
-if applicable, Willow's copy of its repo.
-
-If you're no longer running the version Willow says you've selected, click the
-`Modify?` link to select a different version.
-
-If there are projects where your selected version does _not_ match what Willow
-thinks is latest, they'll show up at the top under the **Outdated projects**
-heading and have a link at the bottom of the card to `View release notes`.
-Clicking that link populates the right column with those release notes.
-
-If there are projects where your selected version _does_ match what Willow
-thinks is latest, they'll show up at the bottom under the **Up-to-date
-projects** heading.
+Note that I still consider the project to be in _alpha_ state. There will be
+bugs; please help fix them!
 
 ## Contributing
 
 Contributions are very much welcome! Please take a look at the [ticket
 tracker][todo] and see if there's anything you're interested in working on. If
 there's specific functionality you'd like to see implemented and it's not
-mentioned in the ticket tracker, please describe it through one of the platforms
-below so we can discuss its inclusion. If we don't feel like it fits with
+mentioned in the ticket tracker, please send a description to the [mailing
+list][email] so we can discuss its inclusion. If I don't feel like it fits with
 Willow's goals, you're encouraged to fork the project and make whatever changes
 you like!
 
-Questions, comments, and patches can always go to the [mailing list][email], but
-there's also an [IRC channel][irc] and an [XMPP MUC][xmpp] for real-time
-interactions.
+Questions, comments, and patches can always be sent to the [mailing
+list][email], but I'm also in the [IRC channel][irc]/[XMPP room][xmpp] pretty
+much 24/7. I might not see messages right away, so please stick around.
 
 - Email: [~amolith/willow@lists.sr.ht][email]
 - IRC: [irc.libera.chat/#willow][irc]

db/project.go

@@ -22,9 +22,9 @@ func DeleteProject(db *sql.DB, mu *sync.Mutex, id string) error {
 }
 
 // GetProject returns a project from the database
-func GetProject(db *sql.DB, id string) (map[string]string, error) {
-	var name, forge, url, version string
-	err := db.QueryRow("SELECT name, forge, url, version FROM projects WHERE id = ?", id).Scan(&name, &forge, &url, &version)
+func GetProject(db *sql.DB, url string) (map[string]string, error) {
+	var id, name, forge, version string
+	err := db.QueryRow("SELECT id, name, forge, version FROM projects WHERE url = ?", url).Scan(&id, &name, &forge, &version)
 	if err != nil {
 		return nil, err
 	}

db/release.go

@@ -9,7 +9,7 @@ import (
 	"sync"
 )
 
-// UpsertRelease adds or updates a release for a project with a given ID in the
+// UpsertRelease adds or updates a release for a project with a given URL in the
 // database
 func UpsertRelease(db *sql.DB, mu *sync.Mutex, id, projectID, url, tag, content, date string) error {
 	mu.Lock()
@@ -49,7 +49,6 @@ func GetReleases(db *sql.DB, projectID string) ([]map[string]string, error) {
 		}
 		releases = append(releases, map[string]string{
 			"id":      id,
-			"project_id": projectID,
 			"url":     url,
 			"tag":     tag,
 			"content": content,

git/git.go

@@ -7,6 +7,7 @@ package git
 import (
 	"errors"
 	"fmt"
+	"io"
 	"net/url"
 	"os"
 	"strings"
@@ -148,23 +149,42 @@ func RemoveRepo(url string) (err error) {
 		return err
 	}
 
-	path = path[:strings.LastIndex(path, "/")]
-	dirs := strings.Split(path, "/")
-
-	for range dirs {
+	// TODO: Check whether the two parent directories are empty and remove them if
+	// so
+	for i := 0; i < 2; i++ {
+		path = strings.TrimSuffix(path, "/")
 		if path == "data" {
 			break
 		}
+		empty, err := dirEmpty(path)
+		if err != nil {
+			return err
+		}
+		if empty {
 			err = os.Remove(path)
 			if err != nil {
-			// This folder likely has data, so might as well save some time by
-			// not checking the parents we can't delete anyway.
-			break
+				return err
+			}
 		}
 		path = path[:strings.LastIndex(path, "/")]
 	}
 
-	return nil
+	return err
+}
+
+// dirEmpty checks if a directory is empty.
+func dirEmpty(name string) (empty bool, err error) {
+	f, err := os.Open(name)
+	if err != nil {
+		return false, err
+	}
+	defer f.Close()
+
+	_, err = f.Readdirnames(1)
+	if err == io.EOF {
+		return true, nil
+	}
+	return false, err
 }
 
 // stringifyRepo accepts a repository URI string and the corresponding local

project/project.go

@@ -7,7 +7,6 @@ package project
 import (
 	"crypto/sha256"
 	"database/sql"
-	"errors"
 	"fmt"
 	"log"
 	"sort"
@@ -42,29 +41,20 @@ type Release struct {
 
 // GetReleases returns a list of all releases for a project from the database
 func GetReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
-	proj.ID = GenProjectID(proj.URL, proj.Name, proj.Forge)
-
 	ret, err := db.GetReleases(dbConn, proj.ID)
 	if err != nil {
 		return proj, err
 	}
 
+	// TODO: figure out a clean way to remove this so the home page loads
+	// immediately.
 	if len(ret) == 0 {
-		proj, err = fetchReleases(dbConn, mu, proj)
-		if err != nil {
-			return proj, err
-		}
-		err = upsertReleases(dbConn, mu, proj.ID, proj.Releases)
-		if err != nil {
-			return proj, err
-		}
 		return proj, nil
 	}
 
 	for _, row := range ret {
 		proj.Releases = append(proj.Releases, Release{
 			ID:      row["id"],
-			ProjectID: proj.ID,
 			Tag:     row["tag"],
 			Content: row["content"],
 			URL:     row["release_url"],
@@ -93,7 +83,7 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
+			err = upsertRelease(dbConn, mu, p.URL, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -112,7 +102,7 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
+			err = upsertRelease(dbConn, mu, p.URL, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -130,18 +120,12 @@ func SortReleases(releases []Release) []Release {
 	return releases
 }
 
-func SortProjects(projects []Project) []Project {
-	sort.Slice(projects, func(i, j int) bool {
-		return strings.ToLower(projects[i].Name) < strings.ToLower(projects[j].Name)
-	})
-	return projects
-}
-
-// upsertReleases updates or inserts a release in the database
-func upsertReleases(dbConn *sql.DB, mu *sync.Mutex, projID string, releases []Release) error {
+// upsertRelease updates or inserts a release in the database
+func upsertRelease(dbConn *sql.DB, mu *sync.Mutex, url string, releases []Release) error {
 	for _, release := range releases {
 		date := release.Date.Format("2006-01-02 15:04:05")
-		err := db.UpsertRelease(dbConn, mu, release.ID, projID, release.URL, release.Tag, release.Content, date)
+		id := GenReleaseID(url, release.URL, release.Tag)
+		err := db.UpsertRelease(dbConn, mu, id, url, release.URL, release.Tag, release.Content, date)
 		if err != nil {
 			log.Printf("Error upserting release: %v", err)
 			return err
@@ -172,19 +156,12 @@ func Track(dbConn *sql.DB, mu *sync.Mutex, manualRefresh *chan struct{}, name, u
 }
 
 func Untrack(dbConn *sql.DB, mu *sync.Mutex, id string) {
-	proj, err := db.GetProject(dbConn, id)
-	if err != nil {
-		fmt.Println("Error getting project:", err)
-	}
-
-	err = db.DeleteProject(dbConn, mu, proj["id"])
+	err := db.DeleteProject(dbConn, mu, id)
 	if err != nil {
 		fmt.Println("Error deleting project:", err)
 	}
 
-	// TODO: before removing, check whether other tracked projects use the same
-	// repo
-	err = git.RemoveRepo(proj["url"])
+	err = git.RemoveRepo(id)
 	if err != nil {
 		log.Println(err)
 	}
@@ -210,7 +187,7 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 			return strings.ToLower(projectsList[i].Name) < strings.ToLower(projectsList[j].Name)
 		})
 		for i := range projectsList {
-			err = upsertReleases(dbConn, mu, projectsList[i].ID, projectsList[i].Releases)
+			err = upsertRelease(dbConn, mu, projectsList[i].URL, projectsList[i].Releases)
 			if err != nil {
 				fmt.Println("Error upserting release:", err)
 				continue
@@ -237,26 +214,24 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 }
 
 // GetProject returns a project from the database
-func GetProject(dbConn *sql.DB, proj Project) (Project, error) {
-	projectDB, err := db.GetProject(dbConn, proj.ID)
-	if err != nil && errors.Is(err, sql.ErrNoRows) {
-		return proj, nil
-	} else if err != nil {
-		return proj, err
+func GetProject(dbConn *sql.DB, id string) (Project, error) {
+	projectDB, err := db.GetProject(dbConn, id)
+	if err != nil {
+		return Project{}, err
 	}
 	p := Project{
-		ID:      proj.ID,
-		URL:     proj.URL,
-		Name:    proj.Name,
-		Forge:   proj.Forge,
+		ID:      projectDB["id"],
+		URL:     projectDB["url"],
+		Name:    projectDB["name"],
+		Forge:   projectDB["forge"],
 		Running: projectDB["version"],
 	}
 	return p, err
 }
 
 // GetProjectWithReleases returns a single project from the database along with its releases
-func GetProjectWithReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
-	project, err := GetProject(dbConn, proj)
+func GetProjectWithReleases(dbConn *sql.DB, mu *sync.Mutex, id string) (Project, error) {
+	project, err := GetProject(dbConn, id)
 	if err != nil {
 		return Project{}, err
 	}
@@ -282,7 +257,7 @@ func GetProjects(dbConn *sql.DB) ([]Project, error) {
 		}
 	}
 
-	return SortProjects(projects), nil
+	return projects, nil
 }
 
 // GetProjectsWithReleases returns a list of all projects and all their releases
@@ -301,5 +276,5 @@ func GetProjectsWithReleases(dbConn *sql.DB, mu *sync.Mutex) ([]Project, error)
 		projects[i].Releases = SortReleases(projects[i].Releases)
 	}
 
-	return SortProjects(projects), nil
+	return projects, nil
 }

ws/static/home.html

@@ -21,11 +21,12 @@
         <header class="wrapper">
             <h1>Willow &nbsp;&nbsp;&nbsp;<span><a href="/logout">Log out</a></span></h1>
             <p><a href="/new">Track a new project</a></p>
+            <p class="message-{{ .MessageType }}">{{ .Message }}</p>
         </header>
         <div class="two_column">
             <div class="projects">
                 <!-- Range through projects that aren't yet up-to-date -->
-                {{- range . -}}
+                {{- range .Projects -}}
                 {{- if ne .Running (index .Releases 0).Tag -}}
                 <h2>Outdated projects</h2>
                 {{- break -}}
@@ -34,7 +35,7 @@
                 {{- range . -}}
                 {{- if ne .Running (index .Releases 0).Tag -}}
                 <div id="{{ .ID }}" class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
+                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h3>
                     <p>You've selected {{ .Running }}. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
                     <p>Latest: <a href="{{ (index .Releases 0).URL }}">{{ (index .Releases 0).Tag }}</a></p>
                     <p><a href="#{{ (index .Releases 0).ID }}">View release notes</a></p>
@@ -43,16 +44,16 @@
                 {{- end -}}
 
                 <!-- Range through projects that _are_ up-to-date -->
-                {{- range . -}}
+                {{- range .Projects -}}
                 {{- if eq .Running (index .Releases 0).Tag -}}
                 <h2>Up-to-date projects</h2>
                 {{- break -}}
                 {{- end -}}
                 {{- end -}}
-                {{- range . -}}
+                {{- range .Projects -}}
                 {{- if eq .Running (index .Releases 0).Tag -}}
                 <div class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
+                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h3>
                     <p>You've selected <a href="#{{ (index .Releases 0).ID }}">{{ .Running }}</a>. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
                 </div>
                 {{- end -}}
@@ -60,7 +61,7 @@
             </div>
             <div class="release_notes">
                 <h2>Release notes</h2>
-                {{- range . -}}
+                {{- range .Projects -}}
                 <div id="{{ (index .Releases 0).ID }}" class="release_note card">
                     <h3>{{ .Name }}: release notes for <a href="{{ (index .Releases 0).URL }}">{{ (index .Releases 0).Tag }}</a> <span class="close"><a href="#">&#x2716;</a></span></h3>
                     {{- if eq .Forge "github" "gitea" "forgejo" -}}

ws/static/select-release.html

@@ -44,7 +44,7 @@
             <input type="hidden" name="name" value="{{ .Name }}">
             <input type="hidden" name="forge" value="{{ .Forge }}">
             <input type="hidden" name="id" value="{{ .ID }}">
-            <input class="button" type="submit" formaction="/new" value="Track releases">
+            <input class="button" type="submit" formaction="/new" value="Track future releases">
         </form>
         <!-- Append these if they ever start limiting RSS entries: `(eq $forge "gitea") (eq $forge "forgejo")` -->
         {{- if or (eq $forge "github") -}}

ws/static/styles.css

@@ -128,7 +128,7 @@ header > h1 > span {
     }
 
     .card {
-        border: 2px solid #424242;
+        border: 2px solid #ccc;
         background: #1c1c1c;
     }
 

ws/ws.go

@@ -29,6 +29,12 @@ type Handler struct {
 	Mu            *sync.Mutex
 }
 
+type page struct {
+	Projects    []project.Project
+	Message     string
+	MessageType string
+}
+
 //go:embed static
 var fs embed.FS
 
@@ -51,7 +57,12 @@ func (h Handler) RootHandler(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 	tmpl := template.Must(template.ParseFS(fs, "static/home.html"))
-	if err := tmpl.Execute(w, data); err != nil {
+	p := page{
+		Projects:    data,
+		Message:     "Hello world",
+		MessageType: "info",
+	}
+	if err := tmpl.Execute(w, p); err != nil {
 		fmt.Println(err)
 	}
 }
@@ -80,6 +91,17 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 
+			proj, err := project.GetProject(h.DbConn, submittedURL)
+			if err != nil && err != sql.ErrNoRows {
+				w.WriteHeader(http.StatusBadRequest)
+				_, err := w.Write([]byte(fmt.Sprintf("Error getting project: %s", err)))
+				if err != nil {
+					fmt.Println(err)
+				}
+				return
+			}
+
+			if proj.Running == "" {
 				forge := bmStrict.Sanitize(params.Get("forge"))
 				if forge == "" {
 					w.WriteHeader(http.StatusBadRequest)
@@ -100,23 +122,17 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 					return
 				}
 
-			proj := project.Project{
-				ID:    project.GenProjectID(submittedURL, name, forge),
+				proj = project.Project{
 					URL:   submittedURL,
 					Name:  name,
 					Forge: forge,
 				}
 
-			proj, err := project.GetProject(h.DbConn, proj)
-			if err != nil && err != sql.ErrNoRows {
-				w.WriteHeader(http.StatusBadRequest)
-				_, err := w.Write([]byte(fmt.Sprintf("Error getting project: %s", err)))
-				if err != nil {
-					fmt.Println(err)
-				}
-				return
+				proj.URL = strings.TrimSuffix(proj.URL, ".git")
+
 			}
 
+			proj.ID = project.GenProjectID(proj.URL, proj.Name, proj.Forge)
 			proj, err = project.GetReleases(h.DbConn, h.Mu, proj)
 			if err != nil {
 				w.WriteHeader(http.StatusBadRequest)