.files/2024-02-24.png.license

@@ -1,3 +0,0 @@
-SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
-
-SPDX-License-Identifier: CC0-1.0

.gitignore

@@ -5,7 +5,7 @@
 /willow
 /*.csv
 /data/
-/*.sqlite*
+/*.sqlite
 /config.toml
 
 /.idea/

README.md

@@ -12,59 +12,41 @@ SPDX-License-Identifier: CC0-1.0
 
 _Forge-agnostic software release tracker_
 
-![screenshot of willow's current web UI](.files/2024-02-24.png)
+![screenshot of willow's current web UI](.files/2023-10-29.png)
 
-_This UI is Amolith's attempt at something simple and functional, yet still
+_This UI is a minimal proof-of-concept, it's going to change drastically in the
-friendly and pleasant. Amolith is not a UX professional and would **very** much
+near future._
-welcome input from one!_
 
 ## What is it?
 
-_If you'd rather watch a short video, Amolith gave a 5-minute [lightning talk on
+_If you'd rather watch a video, I gave a [lightning talk on Willow] at the 2023
-Willow] at the 2023 Ubuntu Summit._
+Ubuntu Summit._
 
 [lightning talk on Willow]: https://youtu.be/XIGxKyekvBQ?t=29900
 
-Willow helps developers, sysadmins, and homelabbers keep up with software
+Willow tracks software releases across arbitrary forge platforms by trying to
-releases across arbitrary forge platforms, including full-featured forges like
+support one of the very few things they all have in common: the VCS. At the
-GitHub, GitLab, or [Forgejo] as well as more minimal options like [cgit] or
+moment, git is the _only_ supported VCS, but I would be interested in adding
-[stagit].
+Pijul, Fossil, Mercurial, etc. You can also track releases using RSS feeds.
 
-[Forgejo]: https://forgejo.org/
+Willow exists because decentralisation can be annoying. One piece of software
-[cgit]: https://git.zx2c4.com/cgit/
+can be found on GitHub, another piece on GitLab, one on Bitbucket, a fourth on
-[stagit]: https://codemadness.org/stagit.html
+SourceHut, and a fifth on the developer's self-hosted Forgejo instance. Forgejo
+and GitHub have RSS feeds that only notify you of releases. GitLab doesn't
+support RSS feeds for anything, just an API you can poke. Some software updates
+might be on the developers' personal blog. Sometimes there are CVEs for specific
+software and they get published somewhere completely different before they're
+fixed in a release.
 
-It exists because decentralisation, as wonderful as it is, does have some pain
+I want to bring all that scattered information under one roof so a developer or
-points. One piece of software is on GitHub, another piece is on GitLab, one on
+sysadmin can pop open willow's web UI and immediately see what needs updating
-Bitbucket, a fourth on [SourceHut], a fifth on the developer's self-hosted
+where. I've recorded some of my other ideas and plans in [my wiki].
-Forgejo instance. The capabilities of each platform can differ as well, making
-the problem even more difficult to solve. Forgejo and GitHub have RSS feeds that
-notify you of releases as well as APIs. SourceHut has both an API and firehose
-RSS feeds that notify you of _all_ activity in the repo. GitLab only has an API.
-Some release announcements might be on the developer's personal blog. Sometimes
-there's a CVE announcement prior to a release and those get published on a
-different platform entirely. It's a mess to keep up with.
 
-[SourceHut]: https://sourcehut.org/
+[my wiki]: https://wiki.secluded.site/hypha/willow
-
-Willow brings some order to that mess by supporting both RSS and one of the
-_very_ few things all the forges have in common: their **V**ersion **C**ontrol
-**S**ystem. At the moment, [Git] is the _only_ supported VCS, but we're
-definitely interested in adding support for [Pijul], [Fossil], [Mercurial], and
-potentially others.
-
-[Git]: https://git-scm.com/
-[Pijul]: https://pijul.org/
-[Fossil]: https://www.fossil-scm.org/
-[Mercurial]: https://www.mercurial-scm.org/
-
-Amolith has recorded some of his other ideas, thoughts, and plans in [his wiki].
-
-[his wiki]: https://wiki.secluded.site/hypha/willow
 
 ## Installation and use
 
-_**Note:** prebuilt binaries will be available after we release [v0.0.1]_
+_**Note:** prebuilt binaries will be available after I release [v0.0.1]_
 
 [v0.0.1]: https://todo.sr.ht/~amolith/willow?search=status%3Aopen%20label%3A%22v0.0.1%22
 
@@ -82,7 +64,7 @@ _**Note:** prebuilt binaries will be available after we release [v0.0.1]_
 * Indicate which version you're currently on
 * That's it!
 
-Note that we still consider the project to be in _alpha_ state. There _will_ be
+Note that I still consider the project to be in _alpha_ state. There will be
 bugs; please help fix them!
 
 ## Contributing
@@ -91,13 +73,13 @@ Contributions are very much welcome! Please take a look at the [ticket
 tracker][todo] and see if there's anything you're interested in working on. If
 there's specific functionality you'd like to see implemented and it's not
 mentioned in the ticket tracker, please send a description to the [mailing
-list][email] so we can discuss its inclusion. If we don't feel like it fits with
+list][email] so we can discuss its inclusion. If I don't feel like it fits with
 Willow's goals, you're encouraged to fork the project and make whatever changes
 you like!
 
-Questions, comments, and patches can always go to the [mailing list][email], but
+Questions, comments, and patches can always be sent to the [mailing
-there's also an [IRC channel][irc] and an [XMPP MUC][xmpp] for real-time
+list][email], but I'm also in the [IRC channel][irc]/[XMPP room][xmpp] pretty
-interactions.
+much 24/7. I might not see messages right away, so please stick around.
 
 - Email: [~amolith/willow@lists.sr.ht][email]
 - IRC: [irc.libera.chat/#willow][irc]
@@ -118,7 +100,6 @@ section._
 
 ``` shell
 git config sendemail.to "~amolith/willow@lists.sr.ht"
-git config format.subjectPrefix "PATCH willow"
 git send-email [HASH]
 ```
 

cmd/willow.go

@@ -10,7 +10,6 @@ import (
 	"log"
 	"net/http"
 	"os"
-	"strconv"
 	"sync"
 
 	"git.sr.ht/~amolith/willow/db"
@@ -24,6 +23,7 @@ import (
 type (
 	Config struct {
 		Server      server
+		CSVLocation string
 		DBConn      string
 		// TODO: Make cache location configurable
 		// CacheLocation string
@@ -90,17 +90,17 @@ func main() {
 		os.Exit(0)
 	}
 
-	mu := sync.Mutex{}
-
 	fmt.Println("Starting refresh loop")
-	go project.RefreshLoop(dbConn, &mu, config.FetchInterval, &manualRefresh, &req, &res)
+	go project.RefreshLoop(dbConn, config.FetchInterval, &manualRefresh, &req, &res)
+
+	var mutex sync.Mutex
 
 	wsHandler := ws.Handler{
 		DbConn:        dbConn,
+		Mutex:         &mutex,
 		Req:           &req,
 		Res:           &res,
 		ManualRefresh: &manualRefresh,
-		Mu:            &mu,
 	}
 
 	mux := http.NewServeMux()
@@ -126,20 +126,6 @@ func main() {
 }
 
 func checkConfig() error {
-	defaultDBConn := "willow.sqlite"
-	defaultFetchInterval := 3600
-	defaultListen := "127.0.0.1:1313"
-
-	defaultConfig := fmt.Sprintf(`# Path to SQLite database
-DBConn = "%s"
-# How often to fetch new releases in seconds
-## Minimum is %ds to avoid rate limits and unintentional abuse
-FetchInterval = %d
-
-[Server]
-# Address to listen on
-Listen = "%s"`, defaultDBConn, defaultFetchInterval, defaultFetchInterval, defaultListen)
-
 	file, err := os.Open(*flagConfig)
 	if err != nil {
 		if os.IsNotExist(err) {
@@ -149,7 +135,15 @@ Listen = "%s"`, defaultDBConn, defaultFetchInterval, defaultFetchInterval, defau
 			}
 			defer file.Close()
 
-			_, err = file.WriteString(defaultConfig)
+			_, err = file.WriteString(`# Path to SQLite database
+DBConn = "willow.sqlite"
+# How often to fetch new releases in seconds
+FetchInterval = 3600
+
+[Server]
+# Address to listen on
+Listen = "127.0.0.1:1313"
+				`)
 			if err != nil {
 				return err
 			}
@@ -168,19 +162,19 @@ Listen = "%s"`, defaultDBConn, defaultFetchInterval, defaultFetchInterval, defau
 		return err
 	}
 
-	if config.FetchInterval < defaultFetchInterval {
+	if config.FetchInterval < 10 {
-		fmt.Println("Fetch interval is set to", strconv.Itoa(config.FetchInterval), "seconds, but the minimum is", defaultFetchInterval, "seconds, using", strconv.Itoa(defaultFetchInterval)+"s")
+		fmt.Println("Fetch interval is set to", config.FetchInterval, "seconds, but the minimum is 10, using 10")
-		config.FetchInterval = defaultFetchInterval
+		config.FetchInterval = 10
 	}
 
 	if config.Server.Listen == "" {
-		fmt.Println("No listen address specified, using", defaultListen)
+		fmt.Println("No listen address specified, using 127.0.0.1:1313")
-		config.Server.Listen = defaultListen
+		config.Server.Listen = "127.0.0.1:1313"
 	}
 
 	if config.DBConn == "" {
-		fmt.Println("No SQLite path specified, using \"" + defaultDBConn + "\"")
+		fmt.Println("No SQLite path specified, using \"willow.sqlite\"")
-		config.DBConn = defaultDBConn
+		config.DBConn = "willow.sqlite"
 	}
 
 	return nil

db/db.go

@@ -7,8 +7,6 @@ package db
 import (
 	"database/sql"
 	_ "embed"
-	"errors"
-	"sync"
 
 	_ "modernc.org/sqlite"
 )
@@ -16,25 +14,46 @@ import (
 //go:embed sql/schema.sql
 var schema string
 
-var mutex = &sync.Mutex{}
-
 // Open opens a connection to the SQLite database
 func Open(dbPath string) (*sql.DB, error) {
-	return sql.Open("sqlite", "file:"+dbPath+"?_pragma=journal_mode%3DWAL")
+	return sql.Open("sqlite", dbPath)
 }
 
 // VerifySchema checks whether the schema has been initalised and initialises it
 // if not
 func InitialiseDatabase(dbConn *sql.DB) error {
 	var name string
-	err := dbConn.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name='users'").Scan(&name)
+	err := dbConn.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name='schema_migrations'").Scan(&name)
-	if err != nil && errors.Is(err, sql.ErrNoRows) {
+	if err == nil {
-		mutex.Lock()
+		return nil
-		defer mutex.Unlock()
+	}
+
+	tables := []string{
+		"users",
+		"sessions",
+		"projects",
+		"releases",
+	}
+
+	for _, table := range tables {
+		name := ""
+		err := dbConn.QueryRow(
+			"SELECT name FROM sqlite_master WHERE type='table' AND name=@table",
+			sql.Named("table", table),
+		).Scan(&name)
+		if err != nil {
+			if err = loadSchema(dbConn); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+// loadSchema loads the initial schema into the database
+func loadSchema(dbConn *sql.DB) error {
 	if _, err := dbConn.Exec(schema); err != nil {
 		return err
 	}
 	return nil
-	}
-	return err
 }

db/migrations.go

@@ -22,10 +22,6 @@ var (
 	migration1Up string
 	//go:embed sql/1_add_project_ids.down.sql
 	migration1Down string
-	//go:embed sql/2_swap_project_url_for_id.up.sql
-	migration2Up string
-	//go:embed sql/2_swap_project_url_for_id.down.sql
-	migration2Down string
 )
 
 var migrations = [...]migration{
@@ -39,13 +35,6 @@ var migrations = [...]migration{
 		downQuery: migration1Down,
 		postHook:  generateAndInsertProjectIDs,
 	},
-	2: {
-		upQuery:   migration2Up,
-		downQuery: migration2Down,
-	},
-	3: {
-		postHook: correctProjectIDs,
-	},
 }
 
 // Migrate runs all pending migrations

db/posthooks.go

@@ -55,35 +55,3 @@ func generateAndInsertProjectIDs(tx *sql.Tx) error {
 
 	return nil
 }
-
-// Basing the project's ID on when it was created (L37) was a bad idea.
-func correctProjectIDs(tx *sql.Tx) error {
-	rows, err := tx.Query("SELECT id, url, name, forge FROM projects")
-	if err != nil {
-		return fmt.Errorf("failed to list projects in projects_tmp: %w", err)
-	}
-	defer rows.Close()
-
-	for rows.Next() {
-		var (
-			old_id string
-			url    string
-			name   string
-			forge  string
-		)
-		if err := rows.Scan(&old_id, &url, &name, &forge); err != nil {
-			return fmt.Errorf("failed to scan row from projects_tmp: %w", err)
-		}
-		id := fmt.Sprintf("%x", sha256.Sum256([]byte(url+name+forge)))
-		_, err = tx.Exec(
-			"UPDATE projects SET id = @id WHERE id = @old_id",
-			sql.Named("id", id),
-			sql.Named("old_id", old_id),
-		)
-		if err != nil {
-			return fmt.Errorf("failed to insert project into projects: %w", err)
-		}
-	}
-
-	return nil
-}

db/project.go

@@ -4,32 +4,32 @@
 
 package db
 
-import (
+import "database/sql"
-	"database/sql"
+
-	"sync"
+// CreateProject adds a project to the database
-)
+func CreateProject(db *sql.DB, url, name, forge, running string) error {
+	_, err := db.Exec("INSERT INTO projects (url, name, forge, version) VALUES (?, ?, ?, ?)", url, name, forge, running)
+	return err
+}
 
 // DeleteProject deletes a project from the database
-func DeleteProject(db *sql.DB, mu *sync.Mutex, id string) error {
+func DeleteProject(db *sql.DB, url string) error {
-	mu.Lock()
+	_, err := db.Exec("DELETE FROM projects WHERE url = ?", url)
-	defer mu.Unlock()
-	_, err := db.Exec("DELETE FROM projects WHERE id = ?", id)
 	if err != nil {
 		return err
 	}
-	_, err = db.Exec("DELETE FROM releases WHERE project_id = ?", id)
+	_, err = db.Exec("DELETE FROM releases WHERE project_url = ?", url)
 	return err
 }
 
 // GetProject returns a project from the database
-func GetProject(db *sql.DB, id string) (map[string]string, error) {
+func GetProject(db *sql.DB, url string) (map[string]string, error) {
-	var name, forge, url, version string
+	var name, forge, version string
-	err := db.QueryRow("SELECT name, forge, url, version FROM projects WHERE id = ?", id).Scan(&name, &forge, &url, &version)
+	err := db.QueryRow("SELECT name, forge, version FROM projects WHERE url = ?", url).Scan(&name, &forge, &version)
 	if err != nil {
 		return nil, err
 	}
 	project := map[string]string{
-		"id":      id,
 		"name":    name,
 		"url":     url,
 		"forge":   forge,
@@ -38,23 +38,27 @@ func GetProject(db *sql.DB, id string) (map[string]string, error) {
 	return project, nil
 }
 
+// UpdateProject updates an existing project in the database
+func UpdateProject(db *sql.DB, url, name, forge, running string) error {
+	_, err := db.Exec("UPDATE projects SET name=?, forge=?, version=? WHERE url=?", name, forge, running, url)
+	return err
+}
+
 // UpsertProject adds or updates a project in the database
-func UpsertProject(db *sql.DB, mu *sync.Mutex, id, url, name, forge, running string) error {
+func UpsertProject(db *sql.DB, url, name, forge, running string) error {
-	mu.Lock()
+	_, err := db.Exec(`INSERT INTO projects (url, name, forge, version)
-	defer mu.Unlock()
+		VALUES (?, ?, ?, ?)
-	_, err := db.Exec(`INSERT INTO projects (id, url, name, forge, version)
+		ON CONFLICT(url) DO 
-		VALUES (?, ?, ?, ?, ?)
-		ON CONFLICT(id) DO 
 			UPDATE SET
 				name = excluded.name,
 				forge = excluded.forge,
-				version = excluded.version;`, id, url, name, forge, running)
+				version = excluded.version;`, url, name, forge, running)
 	return err
 }
 
 // GetProjects returns a list of all projects in the database
 func GetProjects(db *sql.DB) ([]map[string]string, error) {
-	rows, err := db.Query("SELECT id, name, url, forge, version FROM projects")
+	rows, err := db.Query("SELECT name, url, forge, version FROM projects")
 	if err != nil {
 		return nil, err
 	}
@@ -62,13 +66,12 @@ func GetProjects(db *sql.DB) ([]map[string]string, error) {
 
 	var projects []map[string]string
 	for rows.Next() {
-		var id, name, url, forge, version string
+		var name, url, forge, version string
-		err = rows.Scan(&id, &name, &url, &forge, &version)
+		err = rows.Scan(&name, &url, &forge, &version)
 		if err != nil {
 			return nil, err
 		}
 		project := map[string]string{
-			"id":      id,
 			"name":    name,
 			"url":     url,
 			"forge":   forge,

db/release.go

@@ -6,29 +6,34 @@ package db
 
 import (
 	"database/sql"
-	"sync"
 )
 
-// UpsertRelease adds or updates a release for a project with a given ID in the
+// AddRelease adds a release for a project with a given URL to the database
+
+// DeleteRelease deletes a release for a project with a given URL from the database
+
+// UpdateRelease updates a release for a project with a given URL in the database
+
+// UpsertRelease adds or updates a release for a project with a given URL in the
 // database
-func UpsertRelease(db *sql.DB, mu *sync.Mutex, id, projectID, url, tag, content, date string) error {
+func UpsertRelease(db *sql.DB, id, projectURL, releaseURL, tag, content, date string) error {
-	mu.Lock()
+	_, err := db.Exec(`INSERT INTO releases (id, project_url, release_url, tag, content, date)
-	defer mu.Unlock()
-	_, err := db.Exec(`INSERT INTO releases (id, project_id, url, tag, content, date)
 		VALUES (?, ?, ?, ?, ?, ?)
 		ON CONFLICT(id) DO 
 			UPDATE SET
-				url = excluded.url,
+				release_url = excluded.release_url,
 				content = excluded.content,
 				tag = excluded.tag,
 				content = excluded.content,
-				date = excluded.date;`, id, projectID, url, tag, content, date)
+				date = excluded.date;`, id, projectURL, releaseURL, tag, content, date)
 	return err
 }
 
-// GetReleases returns all releases for a project with a given id from the database
+// GetRelease returns a release for a project with a given URL from the database
-func GetReleases(db *sql.DB, projectID string) ([]map[string]string, error) {
+
-	rows, err := db.Query(`SELECT id, url, tag, content, date FROM releases WHERE project_id = ?`, projectID)
+// GetReleases returns all releases for a project with a given URL from the database
+func GetReleases(db *sql.DB, projectURL string) ([]map[string]string, error) {
+	rows, err := db.Query(`SELECT project_url, release_url, tag, content, date FROM releases WHERE project_url = ?`, projectURL)
 	if err != nil {
 		return nil, err
 	}
@@ -37,20 +42,19 @@ func GetReleases(db *sql.DB, projectID string) ([]map[string]string, error) {
 	releases := make([]map[string]string, 0)
 	for rows.Next() {
 		var (
-			id      string
+			projectURL string
-			url     string
+			releaseURL string
 			tag        string
 			content    string
 			date       string
 		)
-		err := rows.Scan(&id, &url, &tag, &content, &date)
+		err := rows.Scan(&projectURL, &releaseURL, &tag, &content, &date)
 		if err != nil {
 			return nil, err
 		}
 		releases = append(releases, map[string]string{
-			"id":         id,
+			"projectURL": projectURL,
-			"project_id": projectID,
+			"releaseURL": releaseURL,
-			"url":        url,
 			"tag":        tag,
 			"content":    content,
 			"date":       date,

db/sql/2_swap_project_url_for_id.down.sql

@@ -1,28 +0,0 @@
--- SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
---
--- SPDX-License-Identifier: CC0-1.0
-
-ALTER TABLE releases RENAME TO releases_tmp;
-
-CREATE TABLE IF NOT EXISTS releases (
-    id          TEXT      NOT NULL PRIMARY KEY,
-    project_url TEXT      NOT NULL,
-    release_url TEXT      NOT NULL,
-    tag         TEXT      NOT NULL,
-    content     TEXT      NOT NULL,
-    date        TIMESTAMP NOT NULL,
-    created_at  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
-);
-
-INSERT INTO releases (id, project_url, release_url, tag, content, date)
-SELECT
-    r.id,
-    p.url,
-    r.url,
-    r.tag,
-    r.content,
-    r.date
-FROM releases_tmp r
-JOIN projects p ON r.project_url = p.url;
-
-DROP TABLE releases_tmp;

db/sql/2_swap_project_url_for_id.up.sql

@@ -1,29 +0,0 @@
--- SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
---
--- SPDX-License-Identifier: CC0-1.0
-
-ALTER TABLE releases RENAME TO releases_tmp;
-
-CREATE TABLE IF NOT EXISTS releases
-(
-    id         TEXT      NOT NULL PRIMARY KEY,
-    project_id TEXT      NOT NULL,
-    url        TEXT      NOT NULL,
-    tag        TEXT      NOT NULL,
-    content    TEXT      NOT NULL,
-    date       TIMESTAMP NOT NULL,
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
-);
-
-INSERT INTO releases (id, project_id, url, tag, content, date)
-SELECT
-    r.id,
-    p.id,
-    r.release_url,
-    r.tag,
-    r.content,
-    r.date
-FROM releases_tmp r
-JOIN projects p ON r.project_url = p.url;
-
-DROP TABLE releases_tmp;

db/users.go

@@ -12,16 +12,12 @@ import (
 // DeleteUser deletes specific user from the database and returns an error if it
 // fails
 func DeleteUser(db *sql.DB, user string) error {
-	mutex.Lock()
-	defer mutex.Unlock()
 	_, err := db.Exec("DELETE FROM users WHERE username = ?", user)
 	return err
 }
 
 // CreateUser creates a new user in the database and returns an error if it fails
 func CreateUser(db *sql.DB, username, hash, salt string) error {
-	mutex.Lock()
-	defer mutex.Unlock()
 	_, err := db.Exec("INSERT INTO users (username, hash, salt) VALUES (?, ?, ?)", username, hash, salt)
 	return err
 }
@@ -76,8 +72,6 @@ func GetSession(db *sql.DB, session string) (string, time.Time, error) {
 // InvalidateSession invalidates a session by setting the expiration date to the
 // provided time.
 func InvalidateSession(db *sql.DB, session string, expiry time.Time) error {
-	mutex.Lock()
-	defer mutex.Unlock()
 	_, err := db.Exec("UPDATE sessions SET expires = ? WHERE token = ?", expiry.Format(time.RFC3339), session)
 	return err
 }
@@ -85,8 +79,6 @@ func InvalidateSession(db *sql.DB, session string, expiry time.Time) error {
 // CreateSession creates a new session in the database and returns an error if
 // it fails
 func CreateSession(db *sql.DB, username, token string, expiry time.Time) error {
-	mutex.Lock()
-	defer mutex.Unlock()
 	_, err := db.Exec("INSERT INTO sessions (token, username, expires) VALUES (?, ?, ?)", token, username, expiry.Format(time.RFC3339))
 	return err
 }

git/git.go

@@ -7,6 +7,7 @@ package git
 import (
 	"errors"
 	"fmt"
+	"io"
 	"net/url"
 	"os"
 	"strings"
@@ -148,31 +149,47 @@ func RemoveRepo(url string) (err error) {
 		return err
 	}
 
-	path = path[:strings.LastIndex(path, "/")]
+	// TODO: Check whether the two parent directories are empty and remove them if
-	dirs := strings.Split(path, "/")
+	// so
-
+	for i := 0; i < 2; i++ {
-	for range dirs {
+		path = strings.TrimSuffix(path, "/")
 		if path == "data" {
 			break
 		}
+		empty, err := dirEmpty(path)
+		if err != nil {
+			return err
+		}
+		if empty {
 			err = os.Remove(path)
 			if err != nil {
-			// This folder likely has data, so might as well save some time by
+				return err
-			// not checking the parents we can't delete anyway.
+			}
-			break
 		}
 		path = path[:strings.LastIndex(path, "/")]
 	}
 
-	return nil
+	return err
+}
+
+// dirEmpty checks if a directory is empty.
+func dirEmpty(name string) (empty bool, err error) {
+	f, err := os.Open(name)
+	if err != nil {
+		return false, err
+	}
+	defer f.Close()
+
+	_, err = f.Readdirnames(1)
+	if err == io.EOF {
+		return true, nil
+	}
+	return false, err
 }
 
 // stringifyRepo accepts a repository URI string and the corresponding local
 // filesystem path, whether the URI is HTTP, HTTPS, or SSH.
 func stringifyRepo(url string) (path string, err error) {
-	url = strings.TrimSuffix(url, ".git")
-	url = strings.TrimSuffix(url, "/")
-
 	ep, err := transport.NewEndpoint(url)
 	if err != nil {
 		return "", err
@@ -181,7 +198,7 @@ func stringifyRepo(url string) (path string, err error) {
 	if ep.Protocol == "http" || ep.Protocol == "https" {
 		return "data/" + strings.Split(url, "://")[1], nil
 	} else if ep.Protocol == "ssh" {
-		return "data/" + ep.Host + "/" + ep.Path, nil
+		return "data/" + ep.Host + ep.Path, nil
 	} else {
 		return "", errors.New("unsupported protocol")
 	}

git/git_test.go

@@ -1,63 +0,0 @@
-// SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
-//
-// SPDX-License-Identifier: Apache-2.0
-
-package git
-
-import (
-	"testing"
-)
-
-func TestStringifyRepo(t *testing.T) {
-	wantGitHub := "data/github.com/owner/repo"
-	wantSourceHut := "data/git.sr.ht/~owner/repo"
-
-	tests := []struct {
-		name  string
-		input string
-		want  string
-	}{
-		{
-			name:  "GitHubHTTP",
-			input: "http://github.com/owner/repo",
-			want:  wantGitHub,
-		},
-		{
-			name:  "GitHubHTTPS",
-			input: "https://github.com/owner/repo",
-			want:  wantGitHub,
-		},
-		{
-			name:  "GitHubSSH",
-			input: "git@github.com:owner/repo",
-			want:  wantGitHub,
-		},
-		{
-			name:  "SourceHutHTTP",
-			input: "http://git.sr.ht/~owner/repo",
-			want:  wantSourceHut,
-		},
-		{
-			name:  "SourceHutHTTPS",
-			input: "https://git.sr.ht/~owner/repo",
-			want:  wantSourceHut,
-		},
-		{
-			name:  "SourceHutSSH",
-			input: "git@git.sr.ht:~owner/repo",
-			want:  wantSourceHut,
-		},
-	}
-
-	for _, test := range tests {
-		t.Run(test.name, func(t *testing.T) {
-			got, err := stringifyRepo(test.input)
-			if err != nil {
-				t.Errorf("stringifyRepo(%s) returned error: %v", test.input, err)
-			}
-			if got != test.want {
-				t.Errorf("stringifyRepo(%s) = %s, want %s", test.input, got, test.want)
-			}
-		})
-	}
-}

go.mod

@@ -10,24 +10,24 @@ toolchain go1.21.3
 
 require (
 	github.com/BurntSushi/toml v1.3.2
-	github.com/go-git/go-git/v5 v5.11.0
+	github.com/go-git/go-git/v5 v5.10.1
 	github.com/microcosm-cc/bluemonday v1.0.26
 	github.com/mmcdole/gofeed v1.2.1
 	github.com/spf13/pflag v1.0.5
-	golang.org/x/crypto v0.19.0
+	golang.org/x/crypto v0.16.0
-	golang.org/x/term v0.17.0
+	golang.org/x/term v0.15.0
 	modernc.org/sqlite v1.27.0
 )
 
 require (
 	dario.cat/mergo v1.0.0 // indirect
 	github.com/Microsoft/go-winio v0.6.1 // indirect
-	github.com/ProtonMail/go-crypto v1.0.0 // indirect
+	github.com/ProtonMail/go-crypto v0.0.0-20230923063757-afb1ddc0824c // indirect
 	github.com/PuerkitoBio/goquery v1.8.1 // indirect
 	github.com/acomagu/bufpipe v1.0.4 // indirect
 	github.com/andybalholm/cascadia v1.3.2 // indirect
 	github.com/aymerick/douceur v0.2.0 // indirect
-	github.com/cloudflare/circl v1.3.7 // indirect
+	github.com/cloudflare/circl v1.3.6 // indirect
 	github.com/cyphar/filepath-securejoin v0.2.4 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
 	github.com/emirpasic/gods v1.18.1 // indirect
@@ -50,11 +50,11 @@ require (
 	github.com/skeema/knownhosts v1.2.1 // indirect
 	github.com/unascribed/FlexVer/go/flexver v1.0.0 // indirect
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
-	golang.org/x/mod v0.15.0 // indirect
+	golang.org/x/mod v0.14.0 // indirect
-	golang.org/x/net v0.21.0 // indirect
+	golang.org/x/net v0.19.0 // indirect
-	golang.org/x/sys v0.17.0 // indirect
+	golang.org/x/sys v0.15.0 // indirect
 	golang.org/x/text v0.14.0 // indirect
-	golang.org/x/tools v0.18.0 // indirect
+	golang.org/x/tools v0.16.0 // indirect
 	gopkg.in/warnings.v0 v0.1.2 // indirect
 	lukechampine.com/uint128 v1.3.0 // indirect
 	modernc.org/cc/v3 v3.41.0 // indirect

go.sum

@@ -7,8 +7,6 @@ github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migc
 github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
 github.com/ProtonMail/go-crypto v0.0.0-20230923063757-afb1ddc0824c h1:kMFnB0vCcX7IL/m9Y5LO+KQYv+t1CQOiFe6+SV2J7bE=
 github.com/ProtonMail/go-crypto v0.0.0-20230923063757-afb1ddc0824c/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
-github.com/ProtonMail/go-crypto v1.0.0 h1:LRuvITjQWX+WIfr930YHG2HNfjR1uOfyf5vE0kC2U78=
-github.com/ProtonMail/go-crypto v1.0.0/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
 github.com/PuerkitoBio/goquery v1.8.1 h1:uQxhNlArOIdbrH1tr0UXwdVFgDcZDrZVdcpygAcwmWM=
 github.com/PuerkitoBio/goquery v1.8.1/go.mod h1:Q8ICL1kNUJ2sXGoAhPGUdYDJvgQgHzJsnnd3H7Ho5jQ=
 github.com/acomagu/bufpipe v1.0.4 h1:e3H4WUzM3npvo5uv95QuJM3cQspFNtFBzvJ2oNjKIDQ=
@@ -26,8 +24,6 @@ github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7N
 github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
 github.com/cloudflare/circl v1.3.6 h1:/xbKIqSHbZXHwkhbrhrt2YOHIwYJlXH94E3tI/gDlUg=
 github.com/cloudflare/circl v1.3.6/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
-github.com/cloudflare/circl v1.3.7 h1:qlCDlTPz2n9fu58M0Nh1J/JzcFpfgkFHHX3O35r5vcU=
-github.com/cloudflare/circl v1.3.7/go.mod h1:sRTcRWXGLrKw6yIGJ+l7amYJFfAXbZG0kBSc8r4zxgA=
 github.com/cyphar/filepath-securejoin v0.2.4 h1:Ugdm7cg7i6ZK6x3xDF1oEu1nfkyfH53EtKeQYTC3kyg=
 github.com/cyphar/filepath-securejoin v0.2.4/go.mod h1:aPGpWjXOXUn2NCNjFvBE6aRxGGx79pTxQpKOJNYHHl4=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -51,8 +47,6 @@ github.com/go-git/go-git/v5 v5.10.0 h1:F0x3xXrAWmhwtzoCokU4IMPcBdncG+HAAqi9FcOOj
 github.com/go-git/go-git/v5 v5.10.0/go.mod h1:1FOZ/pQnqw24ghP2n7cunVl0ON55BsjPYvhWHvZGhoo=
 github.com/go-git/go-git/v5 v5.10.1 h1:tu8/D8i+TWxgKpzQ3Vc43e+kkhXqtsZCKI/egajKnxk=
 github.com/go-git/go-git/v5 v5.10.1/go.mod h1:uEuHjxkHap8kAl//V5F/nNWwqIYtP/402ddd05mp0wg=
-github.com/go-git/go-git/v5 v5.11.0 h1:XIZc1p+8YzypNr34itUfSvYJcv+eYdTnTvOZ2vD3cA4=
-github.com/go-git/go-git/v5 v5.11.0/go.mod h1:6GFcX2P3NM7FPBfpePbpLd21XxsgdAt+lKqXmCUiUCY=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
@@ -124,7 +118,6 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
 github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/unascribed/FlexVer/go/flexver v1.0.0 h1:eaAAWwaT8TiGK75wfEgQRPRVJc1ZIiLTLGUKXxpcs0c=
 github.com/unascribed/FlexVer/go/flexver v1.0.0/go.mod h1:OkWZGfmV3DV2ADlgoS7W1+dD1OOci4mEracZCi3ulBk=
 github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
@@ -139,16 +132,12 @@ golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
 golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
 golang.org/x/crypto v0.16.0 h1:mMMrFzRSCF0GvB7Ne27XVtVAaXLrPmgPC7/v0tkwHaY=
 golang.org/x/crypto v0.16.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
-golang.org/x/crypto v0.19.0 h1:ENy+Az/9Y1vSrlrvBSyna3PITt4tiZLf7sgCjZBX7Wo=
-golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
 golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
 golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
 golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY=
 golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
 golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0=
 golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
-golang.org/x/mod v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
-golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210916014120-12bc252f5db8/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
@@ -163,8 +152,6 @@ golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
 golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
 golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
 golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
-golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4=
-golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -189,8 +176,6 @@ golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
 golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
 golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
-golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
-golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
@@ -201,8 +186,6 @@ golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek=
 golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
 golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4=
 golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
-golang.org/x/term v0.17.0 h1:mkTF7LCd6WGJNL3K1Ad7kwxNfYAW6a8a8QqtMblp/4U=
-golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -223,10 +206,6 @@ golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc=
 golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg=
 golang.org/x/tools v0.16.0 h1:GO788SKMRunPIBCXiQyo2AaexLstOrVhuAL5YwsckQM=
 golang.org/x/tools v0.16.0/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0=
-golang.org/x/tools v0.17.0 h1:FvmRgNOcs3kOa+T20R1uhfP9F6HgG2mfxDv1vrx1Htc=
-golang.org/x/tools v0.17.0/go.mod h1:xsh6VxdV005rRVaS6SSAf9oiAqljS7UZUacMZ8Bnsps=
-golang.org/x/tools v0.18.0 h1:k8NLag8AGHnn+PHbl7g43CtqZAwG60vZkLqgyZgIHgQ=
-golang.org/x/tools v0.18.0/go.mod h1:GL7B4CwcLLeo59yx/9UWWuNOW1n3VZ4f5axWfML7Lcg=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

project/project.go

@@ -7,12 +7,10 @@ package project
 import (
 	"crypto/sha256"
 	"database/sql"
-	"errors"
 	"fmt"
 	"log"
 	"sort"
 	"strings"
-	"sync"
 	"time"
 
 	"github.com/unascribed/FlexVer/go/flexver"
@@ -23,7 +21,6 @@ import (
 )
 
 type Project struct {
-	ID       string
 	URL      string
 	Name     string
 	Forge    string
@@ -33,7 +30,6 @@ type Project struct {
 
 type Release struct {
 	ID      string
-	ProjectID string
 	URL     string
 	Tag     string
 	Content string
@@ -41,30 +37,18 @@ type Release struct {
 }
 
 // GetReleases returns a list of all releases for a project from the database
-func GetReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
+func GetReleases(dbConn *sql.DB, proj Project) (Project, error) {
-	proj.ID = GenProjectID(proj.URL, proj.Name, proj.Forge)
+	ret, err := db.GetReleases(dbConn, proj.URL)
-
-	ret, err := db.GetReleases(dbConn, proj.ID)
 	if err != nil {
 		return proj, err
 	}
 
 	if len(ret) == 0 {
-		proj, err = fetchReleases(dbConn, mu, proj)
+		return fetchReleases(dbConn, proj)
-		if err != nil {
-			return proj, err
-		}
-		err = upsertReleases(dbConn, mu, proj.ID, proj.Releases)
-		if err != nil {
-			return proj, err
-		}
-		return proj, nil
 	}
 
 	for _, row := range ret {
 		proj.Releases = append(proj.Releases, Release{
-			ID:        row["id"],
-			ProjectID: proj.ID,
 			Tag:     row["tag"],
 			Content: row["content"],
 			URL:     row["release_url"],
@@ -76,7 +60,7 @@ func GetReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error)
 }
 
 // fetchReleases fetches releases from a project's forge given its URI
-func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
+func fetchReleases(dbConn *sql.DB, p Project) (Project, error) {
 	var err error
 	switch p.Forge {
 	case "github", "gitea", "forgejo":
@@ -87,13 +71,13 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 		}
 		for _, release := range rssReleases {
 			p.Releases = append(p.Releases, Release{
-				ID:      GenReleaseID(p.URL, release.URL, release.Tag),
+				ID:      genReleaseID(p.URL, release.URL, release.Tag),
 				Tag:     release.Tag,
 				Content: release.Content,
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
+			err = upsert(dbConn, p.URL, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -106,13 +90,13 @@ func fetchReleases(dbConn *sql.DB, mu *sync.Mutex, p Project) (Project, error) {
 		}
 		for _, release := range gitReleases {
 			p.Releases = append(p.Releases, Release{
-				ID:      GenReleaseID(p.URL, release.URL, release.Tag),
+				ID:      genReleaseID(p.URL, release.URL, release.Tag),
 				Tag:     release.Tag,
 				Content: release.Content,
 				URL:     release.URL,
 				Date:    release.Date,
 			})
-			err = upsertReleases(dbConn, mu, p.ID, p.Releases)
+			err = upsert(dbConn, p.URL, p.Releases)
 			if err != nil {
 				log.Printf("Error upserting release: %v", err)
 				return p, err
@@ -130,18 +114,12 @@ func SortReleases(releases []Release) []Release {
 	return releases
 }
 
-func SortProjects(projects []Project) []Project {
+// upsert updates or inserts a project release into the database
-	sort.Slice(projects, func(i, j int) bool {
+func upsert(dbConn *sql.DB, url string, releases []Release) error {
-		return strings.ToLower(projects[i].Name) < strings.ToLower(projects[j].Name)
-	})
-	return projects
-}
-
-// upsertReleases updates or inserts a release in the database
-func upsertReleases(dbConn *sql.DB, mu *sync.Mutex, projID string, releases []Release) error {
 	for _, release := range releases {
 		date := release.Date.Format("2006-01-02 15:04:05")
-		err := db.UpsertRelease(dbConn, mu, release.ID, projID, release.URL, release.Tag, release.Content, date)
+		id := genReleaseID(url, release.URL, release.Tag)
+		err := db.UpsertRelease(dbConn, id, url, release.URL, release.Tag, release.Content, date)
 		if err != nil {
 			log.Printf("Error upserting release: %v", err)
 			return err
@@ -150,47 +128,34 @@ func upsertReleases(dbConn *sql.DB, mu *sync.Mutex, projID string, releases []Re
 	return nil
 }
 
-// GenReleaseID generates a likely-unique ID from its project's URL, its release's URL, and its tag
+func genReleaseID(projectURL, releaseURL, tag string) string {
-func GenReleaseID(projectURL, releaseURL, tag string) string {
 	idByte := sha256.Sum256([]byte(projectURL + releaseURL + tag))
 	return fmt.Sprintf("%x", idByte)
 }
 
-// GenProjectID generates a likely-unique ID from a project's URI, name, and forge
+func Track(dbConn *sql.DB, manualRefresh *chan struct{}, name, url, forge, release string) {
-func GenProjectID(url, name, forge string) string {
+	err := db.UpsertProject(dbConn, url, name, forge, release)
-	idByte := sha256.Sum256([]byte(url + name + forge))
-	return fmt.Sprintf("%x", idByte)
-}
-
-func Track(dbConn *sql.DB, mu *sync.Mutex, manualRefresh *chan struct{}, name, url, forge, release string) {
-	id := GenProjectID(url, name, forge)
-	err := db.UpsertProject(dbConn, mu, id, url, name, forge, release)
 	if err != nil {
 		fmt.Println("Error upserting project:", err)
 	}
 	*manualRefresh <- struct{}{}
 }
 
-func Untrack(dbConn *sql.DB, mu *sync.Mutex, id string) {
+func Untrack(dbConn *sql.DB, manualRefresh *chan struct{}, url string) {
-	proj, err := db.GetProject(dbConn, id)
+	err := db.DeleteProject(dbConn, url)
-	if err != nil {
-		fmt.Println("Error getting project:", err)
-	}
-
-	err = db.DeleteProject(dbConn, mu, proj["id"])
 	if err != nil {
 		fmt.Println("Error deleting project:", err)
 	}
 
-	// TODO: before removing, check whether other tracked projects use the same
+	*manualRefresh <- struct{}{}
-	// repo
+
-	err = git.RemoveRepo(proj["url"])
+	err = git.RemoveRepo(url)
 	if err != nil {
 		log.Println(err)
 	}
 }
 
-func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, req *chan struct{}, res *chan []Project) {
+func RefreshLoop(dbConn *sql.DB, interval int, manualRefresh, req *chan struct{}, res *chan []Project) {
 	ticker := time.NewTicker(time.Second * time.Duration(interval))
 
 	fetch := func() []Project {
@@ -199,7 +164,7 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 			fmt.Println("Error getting projects:", err)
 		}
 		for i, p := range projectsList {
-			p, err := fetchReleases(dbConn, mu, p)
+			p, err := fetchReleases(dbConn, p)
 			if err != nil {
 				fmt.Println(err)
 				continue
@@ -210,7 +175,7 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 			return strings.ToLower(projectsList[i].Name) < strings.ToLower(projectsList[j].Name)
 		})
 		for i := range projectsList {
-			err = upsertReleases(dbConn, mu, projectsList[i].ID, projectsList[i].Releases)
+			err = upsert(dbConn, projectsList[i].URL, projectsList[i].Releases)
 			if err != nil {
 				fmt.Println("Error upserting release:", err)
 				continue
@@ -237,31 +202,18 @@ func RefreshLoop(dbConn *sql.DB, mu *sync.Mutex, interval int, manualRefresh, re
 }
 
 // GetProject returns a project from the database
-func GetProject(dbConn *sql.DB, proj Project) (Project, error) {
+func GetProject(dbConn *sql.DB, url string) (Project, error) {
-	projectDB, err := db.GetProject(dbConn, proj.ID)
+	projectDB, err := db.GetProject(dbConn, url)
-	if err != nil && errors.Is(err, sql.ErrNoRows) {
-		return proj, nil
-	} else if err != nil {
-		return proj, err
-	}
-	p := Project{
-		ID:      proj.ID,
-		URL:     proj.URL,
-		Name:    proj.Name,
-		Forge:   proj.Forge,
-		Running: projectDB["version"],
-	}
-	return p, err
-}
-
-// GetProjectWithReleases returns a single project from the database along with its releases
-func GetProjectWithReleases(dbConn *sql.DB, mu *sync.Mutex, proj Project) (Project, error) {
-	project, err := GetProject(dbConn, proj)
 	if err != nil {
 		return Project{}, err
 	}
-
+	p := Project{
-	return GetReleases(dbConn, mu, project)
+		URL:     projectDB["url"],
+		Name:    projectDB["name"],
+		Forge:   projectDB["forge"],
+		Running: projectDB["version"],
+	}
+	return p, err
 }
 
 // GetProjects returns a list of all projects from the database
@@ -274,7 +226,6 @@ func GetProjects(dbConn *sql.DB) ([]Project, error) {
 	projects := make([]Project, len(projectsDB))
 	for i, p := range projectsDB {
 		projects[i] = Project{
-			ID:      p["id"],
 			URL:     p["url"],
 			Name:    p["name"],
 			Forge:   p["forge"],
@@ -282,24 +233,5 @@ func GetProjects(dbConn *sql.DB) ([]Project, error) {
 		}
 	}
 
-	return SortProjects(projects), nil
+	return projects, nil
-}
-
-// GetProjectsWithReleases returns a list of all projects and all their releases
-// from the database
-func GetProjectsWithReleases(dbConn *sql.DB, mu *sync.Mutex) ([]Project, error) {
-	projects, err := GetProjects(dbConn)
-	if err != nil {
-		return nil, err
-	}
-
-	for i := range projects {
-		projects[i], err = GetReleases(dbConn, mu, projects[i])
-		if err != nil {
-			return nil, err
-		}
-		projects[i].Releases = SortReleases(projects[i].Releases)
-	}
-
-	return SortProjects(projects), nil
 }

ws/static/home.html

@@ -18,51 +18,19 @@
         <link rel="stylesheet" href="/static/styles.css" />
     </head>
     <body>
-        <header class="wrapper">
         <h1>Willow &nbsp;&nbsp;&nbsp;<span><a href="/logout">Log out</a></span></h1>
         <p><a href="/new">Track a new project</a></p>
-        </header>
-        <div class="two_column">
         <div class="projects">
         <!-- Range through projects that aren't yet up-to-date -->
         {{- range . -}}
         {{- if ne .Running (index .Releases 0).Tag -}}
-                <h2>Outdated projects</h2>
+        <div class="project">
-                {{- break -}}
+            <h2><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h2>
-                {{- end -}}
-                {{- end -}}
-                {{- range . -}}
-                {{- if ne .Running (index .Releases 0).Tag -}}
-                <div id="{{ .ID }}" class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
             <p>You've selected {{ .Running }}. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
             <p>Latest: <a href="{{ (index .Releases 0).URL }}">{{ (index .Releases 0).Tag }}</a></p>
-                    <p><a href="#{{ (index .Releases 0).ID }}">View release notes</a></p>
+            <p>
-                </div>
+                <details>
-                {{- end -}}
+                    <summary>Expand release notes</summary>
-                {{- end -}}
-
-                <!-- Range through projects that _are_ up-to-date -->
-                {{- range . -}}
-                {{- if eq .Running (index .Releases 0).Tag -}}
-                <h2>Up-to-date projects</h2>
-                {{- break -}}
-                {{- end -}}
-                {{- end -}}
-                {{- range . -}}
-                {{- if eq .Running (index .Releases 0).Tag -}}
-                <div class="project card">
-                    <h3><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span class="delete"><a href="/new?action=delete&id={{ .ID }}">Delete?</a></span></h3>
-                    <p>You've selected <a href="#{{ (index .Releases 0).ID }}">{{ .Running }}</a>. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
-                </div>
-                {{- end -}}
-                {{- end -}}
-            </div>
-            <div class="release_notes">
-                <h2>Release notes</h2>
-                {{- range . -}}
-                <div id="{{ (index .Releases 0).ID }}" class="release_note card">
-                    <h3>{{ .Name }}: release notes for <a href="{{ (index .Releases 0).URL }}">{{ (index .Releases 0).Tag }}</a> <span class="close"><a href="#">&#x2716;</a></span></h3>
                     {{- if eq .Forge "github" "gitea" "forgejo" -}}
                     {{- (index .Releases 0).Content -}}
                     {{- else -}}
@@ -70,10 +38,20 @@
                     {{- (index .Releases 0).Content -}}
                     </pre>
                     {{- end -}}
-                    <p><a class="return_to_project" href="#{{ .ID }}">Back to project</a></p>
+                </details>
+            </p>
         </div>
         {{- end -}}
+        {{- end -}}
+
+        <!-- Range through projects that _are_ up-to-date -->
+        {{- range . -}}
+        {{- if eq .Running (index .Releases 0).Tag -}}
+        <div class="project">
+            <h2><a href="{{ .URL }}">{{ .Name }}</a>&nbsp;&nbsp;&nbsp;<span style="font-size: 12px;"><a href="/new?action=delete&url={{ .URL }}">Delete?</a></span></h2>
+            <p>You've selected {{ .Running }}. <a href="/new?action=update&url={{ .URL }}&forge={{ .Forge }}&name={{ .Name }}">Modify?</a></p>
         </div>
-        </div>
+        {{- end -}}
+        {{- end -}}
     </body>
 </html>

ws/static/login.html

@@ -17,7 +17,7 @@
         <link rel="preload" href="/static/styles.css" as="style" />
         <link rel="stylesheet" href="/static/styles.css" />
     </head>
-    <body class="wrapper">
+    <body>
         <h1>Willow</h1>
         <form method="post">
             <div class="input">

ws/static/new.html

@@ -17,7 +17,7 @@
         <link rel="preload" href="/static/styles.css" as="style" />
         <link rel="stylesheet" href="/static/styles.css" />
     </head>
-    <body class="wrapper">
+    <body>
         <h1>Willow</h1>
         <form method="post">
             <div class="input">

ws/static/select-release.html

@@ -17,7 +17,7 @@
         <link rel="preload" href="/static/styles.css" as="style" />
         <link rel="stylesheet" href="/static/styles.css" />
     </head>
-    <body class="wrapper">
+    <body>
         <h1>Willow</h1>
         <form method="post">
             <div class="input">
@@ -31,9 +31,9 @@
                 <label for="{{ .Tag }}"><a href="{{ .URL }}">{{ .Tag }}</a></label><br>
                 {{- else -}}
                 {{- if eq $forge "sourcehut" -}}
-                <label for="{{ .Tag }}"><a href="{{ $url }}/refs/{{ .Tag }}">{{ .Tag }}</a></label><br>
+                <label for="{{ .Tag }}"><a href="{{ $url }}/refs/{{ .Tag }}">{{ .Tag }}</label><br>
                 {{- else if eq $forge "gitlab" -}}
-                <label for="{{ .Tag }}"><a href="{{ $url }}/-releases/{{ .Tag }}">{{ .Tag }}</a></label><br>
+                <label for="{{ .Tag }}"><a href="{{ $url }}/-releases/{{ .Tag }}">{{ .Tag }}</label><br>
                 {{- else -}}
                 <label for="{{ .Tag }}">{{ .Tag }}</label><br>
                 {{- end -}}
@@ -43,7 +43,6 @@
             <input type="hidden" name="url" value="{{ .URL }}">
             <input type="hidden" name="name" value="{{ .Name }}">
             <input type="hidden" name="forge" value="{{ .Forge }}">
-            <input type="hidden" name="id" value="{{ .ID }}">
             <input class="button" type="submit" formaction="/new" value="Track future releases">
         </form>
         <!-- Append these if they ever start limiting RSS entries: `(eq $forge "gitea") (eq $forge "forgejo")` -->

ws/static/styles.css

@@ -37,11 +37,12 @@
 }
 
 html {
+    max-width: 500px;
     margin: auto auto;
     color: #2f2f2f;
     background: white;
     font-family: 'Atkinson Hyperlegible', sans-serif;
-    scroll-behavior: smooth;
+    padding-bottom: 20px;
 }
 
 a {
@@ -52,28 +53,8 @@ a:visited {
     color: #0640e0;
 }
 
-.two_column {
+.project {
-    display: flex;
+    max-width: 500px;
-    gap: 30px;
-    flex-direction: row;
-    margin: auto auto;
-    max-width: 1000px;
-    height: 92vh;
-}
-
-.projects, .release_notes {
-    overflow: scroll;
-    flex: 0 0 500px;
-}
-
-.release_note.card:not(:target) { display: none;  }
-.release_note.card:target       { display: block; }
-
-.return_to_project {
-    display: none;
-}
-
-.card {
     border: 2px solid #2f2f2f;
     background: #f8f8f8;
     border-radius: 5px;
@@ -82,37 +63,36 @@ a:visited {
     box-shadow: 0 4px 8px 0 rgba(0,0,0,0.2),0 6px 20px 0 rgba(0,0,0,0.19);
 }
 
-.card > h3 {
+.project > h2 {
     margin-top: 0;
 }
 
-.card > p:first-of-type {
+.project > p:first-of-type {
     margin-bottom: 16px;
 }
 
-.card > p:last-of-type {
+.project > h2 > span {
-    margin-bottom: 16px;
-}
-
-.close, .delete { float: right; }
-.delete { font-size: 12px; }
-.close > a {
-    text-decoration: none;
-    color: #2f2f2f;
-}
-
-.card > pre, .card > div > pre { overflow: scroll; }
-
-.wrapper {
-    max-width: 500px;
-    margin: auto auto;
-}
-
-header > h1 > span {
     font-size: 12px;
     float: right;
 }
 
+body > h1 > span {
+    font-size: 12px;
+    float: right;
+}
+
+.project > details > pre {
+    overflow: scroll;
+}
+
+summary {
+    cursor: pointer;
+}
+
+details summary > * {
+    display: inline;
+}
+
 @media (prefers-color-scheme: dark) {
     html {
         background: #171717;
@@ -127,36 +107,8 @@ header > h1 > span {
         color: #5582ff;
     }
 
-    .card {
+    .project {
-        border: 2px solid #424242;
+        border: 2px solid #ccc;
         background: #1c1c1c;
     }
-
-    .close > a {
-        color: #ccc;
-    }
 }
-
-@media only screen and (max-width: 1000px) {
-    div[id] {
-        display: block;
-    }
-
-    .two_column {
-        flex-direction: column;
-    }
-
-    .projects, .release_notes {
-        overflow: visible;
-        flex: 0 0 100%;
-    }
-
-    .return_to_project {
-        display: block;
-    }
-
-    .close {
-        display: none;
-    }
-}
-

ws/ws.go

@@ -16,17 +16,18 @@ import (
 	"text/template"
 	"time"
 
-	"git.sr.ht/~amolith/willow/project"
 	"git.sr.ht/~amolith/willow/users"
+
+	"git.sr.ht/~amolith/willow/project"
 	"github.com/microcosm-cc/bluemonday"
 )
 
 type Handler struct {
 	DbConn        *sql.DB
+	Mutex         *sync.Mutex
 	Req           *chan struct{}
 	ManualRefresh *chan struct{}
 	Res           *chan []project.Project
-	Mu            *sync.Mutex
 }
 
 //go:embed static
@@ -40,16 +41,8 @@ func (h Handler) RootHandler(w http.ResponseWriter, r *http.Request) {
 		http.Redirect(w, r, "/login", http.StatusSeeOther)
 		return
 	}
-	data, err := project.GetProjectsWithReleases(h.DbConn, h.Mu)
+	*h.Req <- struct{}{}
-	if err != nil {
+	data := <-*h.Res
-		fmt.Println(err)
-		w.WriteHeader(http.StatusInternalServerError)
-		_, err := w.Write([]byte("Internal Server Error"))
-		if err != nil {
-			fmt.Println(err)
-		}
-		return
-	}
 	tmpl := template.Must(template.ParseFS(fs, "static/home.html"))
 	if err := tmpl.Execute(w, data); err != nil {
 		fmt.Println(err)
@@ -80,6 +73,17 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 
+			proj, err := project.GetProject(h.DbConn, submittedURL)
+			if err != nil && err != sql.ErrNoRows {
+				w.WriteHeader(http.StatusBadRequest)
+				_, err := w.Write([]byte(fmt.Sprintf("Error getting project: %s", err)))
+				if err != nil {
+					fmt.Println(err)
+				}
+				return
+			}
+
+			if proj.Running == "" {
 				forge := bmStrict.Sanitize(params.Get("forge"))
 				if forge == "" {
 					w.WriteHeader(http.StatusBadRequest)
@@ -100,24 +104,17 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 					return
 				}
 
-			proj := project.Project{
+				proj = project.Project{
-				ID:    project.GenProjectID(submittedURL, name, forge),
 					URL:   submittedURL,
 					Name:  name,
 					Forge: forge,
 				}
 
-			proj, err := project.GetProject(h.DbConn, proj)
+				proj.URL = strings.TrimSuffix(proj.URL, ".git")
-			if err != nil && err != sql.ErrNoRows {
+
-				w.WriteHeader(http.StatusBadRequest)
-				_, err := w.Write([]byte(fmt.Sprintf("Error getting project: %s", err)))
-				if err != nil {
-					fmt.Println(err)
-				}
-				return
 			}
 
-			proj, err = project.GetReleases(h.DbConn, h.Mu, proj)
+			proj, err = project.GetReleases(h.DbConn, proj)
 			if err != nil {
 				w.WriteHeader(http.StatusBadRequest)
 				_, err := w.Write([]byte(fmt.Sprintf("Error getting releases: %s", err)))
@@ -132,8 +129,8 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				fmt.Println(err)
 			}
 		} else if action == "delete" {
-			submittedID := params.Get("id")
+			submittedURL := params.Get("url")
-			if submittedID == "" {
+			if submittedURL == "" {
 				w.WriteHeader(http.StatusBadRequest)
 				_, err := w.Write([]byte("No URL provided"))
 				if err != nil {
@@ -142,7 +139,7 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 				return
 			}
 
-			project.Untrack(h.DbConn, h.Mu, submittedID)
+			project.Untrack(h.DbConn, h.ManualRefresh, submittedURL)
 			http.Redirect(w, r, "/", http.StatusSeeOther)
 		}
 	}
@@ -152,31 +149,30 @@ func (h Handler) NewHandler(w http.ResponseWriter, r *http.Request) {
 		if err != nil {
 			fmt.Println(err)
 		}
-		idValue := bmStrict.Sanitize(r.FormValue("id"))
 		nameValue := bmStrict.Sanitize(r.FormValue("name"))
 		urlValue := bmStrict.Sanitize(r.FormValue("url"))
 		forgeValue := bmStrict.Sanitize(r.FormValue("forge"))
 		releaseValue := bmStrict.Sanitize(r.FormValue("release"))
 
-		// If releaseValue is not empty, we're updating an existing project
+		if nameValue != "" && urlValue != "" && forgeValue != "" && releaseValue != "" {
-		if idValue != "" && nameValue != "" && urlValue != "" && forgeValue != "" && releaseValue != "" {
+			project.Track(h.DbConn, h.ManualRefresh, nameValue, urlValue, forgeValue, releaseValue)
-			project.Track(h.DbConn, h.Mu, h.ManualRefresh, nameValue, urlValue, forgeValue, releaseValue)
 			http.Redirect(w, r, "/", http.StatusSeeOther)
 			return
 		}
 
-		// If releaseValue is empty, we're creating a new project
+		if nameValue != "" && urlValue != "" && forgeValue != "" && releaseValue == "" {
-		if idValue == "" && nameValue != "" && urlValue != "" && forgeValue != "" && releaseValue == "" {
 			http.Redirect(w, r, "/new?action=yoink&name="+url.QueryEscape(nameValue)+"&url="+url.QueryEscape(urlValue)+"&forge="+url.QueryEscape(forgeValue), http.StatusSeeOther)
 			return
 		}
 
+		if nameValue == "" && urlValue == "" && forgeValue == "" && releaseValue == "" {
 			w.WriteHeader(http.StatusBadRequest)
-		_, err = w.Write([]byte("No data provided"))
+			_, err := w.Write([]byte("No data provided"))
 			if err != nil {
 				fmt.Println(err)
 			}
 		}
+	}
 }
 
 func (h Handler) LoginHandler(w http.ResponseWriter, r *http.Request) {