diff --git a/docs/superpowers/plans/2026-05-06-allanime-decomposition.md b/docs/superpowers/plans/2026-05-06-allanime-decomposition.md deleted file mode 100644 index 8234dfa..0000000 --- a/docs/superpowers/plans/2026-05-06-allanime-decomposition.md +++ /dev/null @@ -1,167 +0,0 @@ -# AllAnime Client Decomposition Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Decompose the monolithic `api/playback/allanime_client.go` into focused, maintainable units. - -**Architecture:** -1. Move uTLS networking logic to `pkg/net/utls`. -2. Move crypto and key discovery to `api/playback/allanime_crypto.go`. -3. Move data extraction and stream detection to `api/playback/allanime_extractor.go`. -4. Keep core API and GraphQL logic in `api/playback/allanime_client.go`. - -**Tech Stack:** Go (uTLS, SHA256, AES-CTR) - ---- - -### Task 1: Move uTLS RoundTripper to pkg/net/utls - -**Files:** -- Create: `pkg/net/utls/utls.go` -- Modify: `api/playback/allanime_client.go` - -- [ ] **Step 1: Create pkg/net/utls/utls.go** - -```go -package utls - -import ( - "bufio" - "context" - "net" - "net/http" - "time" - - utls "github.com/refraction-networking/utls" - "golang.org/x/net/http2" -) - -type UtlsRoundTripper struct{} - -func (rt *UtlsRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) { - port := req.URL.Port() - if port == "" { - if req.URL.Scheme == "https" { - port = "443" - } else { - port = "80" - } - } - - addr := net.JoinHostPort(req.URL.Hostname(), port) - conn, err := net.DialTimeout("tcp", addr, 10*time.Second) - if err != nil { - return nil, err - } - - config := &utls.Config{ServerName: req.URL.Hostname()} - uConn := utls.UClient(conn, config, utls.HelloFirefox_Auto) - - if err := uConn.Handshake(); err != nil { - return nil, err - } - - var httpRT http.RoundTripper - switch uConn.ConnectionState().NegotiatedProtocol { - case http2.NextProtoTLS: - t2 := &http2.Transport{ - DialTLSContext: func(ctx context.Context, network, addr string, cfg *utls.Config) (net.Conn, error) { - return uConn, nil - }, - } - httpRT = t2 - default: - t1 := &http.Transport{ - DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) { - return uConn, nil - }, - } - httpRT = t1 - } - - return httpRT.RoundTrip(req) -} -``` - -- [ ] **Step 2: Update allanime_client.go to use the new package** - -Remove `utlsRoundTripper` from `api/playback/allanime_client.go` and update `newAllAnimeClient` to use `&utls.UtlsRoundTripper{}`. - -- [ ] **Step 3: Verify build and commit** - -Run: `go build ./api/playback/...` -Expected: PASS - -```bash -git add pkg/net/utls/utls.go api/playback/allanime_client.go -git commit -m "refactor: move utls roundtripper to pkg/net/utls" -``` - -### Task 2: Extract Crypto and Key Discovery - -**Files:** -- Create: `api/playback/allanime_crypto.go` -- Modify: `api/playback/allanime_client.go` - -- [ ] **Step 1: Move crypto logic to allanime_crypto.go** - -Extract constants (`allAnimeAESKey`, `aniCliRawSourceURL`, etc.) and functions (`decryptTobeparsed`, `tryDecryptCTR`, `getAESKey`, `fetchKeyFromForks`, `extractKey`, `getTestPayload`, `validateKeys`, `getAllKeys`) to `api/playback/allanime_crypto.go`. - -- [ ] **Step 2: Update allanime_client.go** - -Remove the extracted code from `api/playback/allanime_client.go`. - -- [ ] **Step 3: Verify build and commit** - -Run: `go build ./api/playback/...` -Expected: PASS - -```bash -git add api/playback/allanime_crypto.go api/playback/allanime_client.go -git commit -m "refactor: extract allanime crypto and key discovery" -``` - -### Task 3: Extract Data Extraction and Stream Detection - -**Files:** -- Create: `api/playback/allanime_extractor.go` -- Modify: `api/playback/allanime_client.go` - -- [ ] **Step 1: Move extraction logic to allanime_extractor.go** - -Extract functions (`extractSourceURLsFromData`, `buildStreamSource`, `sourceReference` struct, `buildSourceReferences`, `extractEpisodeData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, `getMapKeys`) to `api/playback/allanime_extractor.go`. - -- [ ] **Step 2: Update allanime_client.go** - -Remove the extracted code from `api/playback/allanime_client.go`. Ensure `allAnimeClient` methods still call these (now package-level) functions. - -- [ ] **Step 3: Verify build and commit** - -Run: `go build ./api/playback/...` -Expected: PASS - -```bash -git add api/playback/allanime_extractor.go api/playback/allanime_client.go -git commit -m "refactor: extract allanime data extraction logic" -``` - -### Task 4: Final Cleanup and Verification - -**Files:** -- Modify: `api/playback/allanime_client.go` - -- [ ] **Step 1: Final review of allanime_client.go** - -Ensure only `allAnimeClient` struct, `newAllAnimeClient`, `graphqlRequest`, `graphqlRequestWithHash`, `Search`, `GetEpisodes`, `GetAvailableEpisodes`, and `GetEpisodeMetadata` remain. - -- [ ] **Step 2: Run all tests** - -Run: `go test -v ./api/playback/...` -Expected: PASS - -- [ ] **Step 3: Commit final cleanup** - -```bash -git add api/playback/allanime_client.go -git commit -m "refactor: final cleanup of allanime_client.go" -``` diff --git a/docs/superpowers/plans/2026-05-06-extract-allanime-extraction.md b/docs/superpowers/plans/2026-05-06-extract-allanime-extraction.md deleted file mode 100644 index b46e43f..0000000 --- a/docs/superpowers/plans/2026-05-06-extract-allanime-extraction.md +++ /dev/null @@ -1,300 +0,0 @@ -# Extract AllAnime Data Extraction Logic Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Extract data parsing and stream detection logic from `allanime_client.go` into a new file `allanime_extractor.go` to improve maintainability and separate concerns. - -**Architecture:** Move non-network utility functions and helper structs related to parsing AllAnime responses and URL decoding/detection into a dedicated extractor file within the same package. - -**Tech Stack:** Go (Standard Library) - ---- - -### Task 1: Create allanime_extractor.go - -**Files:** -- Create: `api/playback/allanime_extractor.go` - -- [ ] **Step 1: Create the file with moved logic** - -```go -package playback - -import ( - "context" - "encoding/json" - "fmt" - "log" - "net/url" - "strings" -) - -type sourceReference struct { - URL string - Name string -} - -func (c *allAnimeClient) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource { - episodeData, ok := data["episode"].(map[string]any) - if !ok { - return nil - } - - sourceURLs, ok := episodeData["sourceUrls"].([]any) - if !ok || len(sourceURLs) == 0 { - return nil - } - - references := buildSourceReferences(sourceURLs) - if len(references) == 0 { - return nil - } - - out := make([]StreamSource, 0, len(references)) - for _, ref := range references { - target := strings.TrimSpace(ref.URL) - if target == "" { - continue - } - - if strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") { - sourceType := detectStreamType(target) - if sourceType == "unknown" { - sourceType = detectEmbedType(target) - } - - out = append(out, buildStreamSource(target, sourceType, ref.Name)) - continue - } - - decoded := decodeSourceURL(target) - if decoded == "" { - continue - } - - if strings.HasPrefix(decoded, "http://") || strings.HasPrefix(decoded, "https://") { - sourceType := detectStreamType(decoded) - if sourceType == "unknown" { - sourceType = detectEmbedType(decoded) - } - - out = append(out, buildStreamSource(decoded, sourceType, ref.Name)) - continue - } - - if !strings.HasPrefix(decoded, "/") { - decoded = "/" + decoded - } - - extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded) - if err != nil { - log.Printf("source extraction failed for %s: %v", decoded, err) - continue - } - - out = append(out, extracted...) - } - - return out -} - -func buildStreamSource(url, sourceType, provider string) StreamSource { - return StreamSource{ - URL: url, - Provider: provider, - Type: sourceType, - Referer: allAnimeReferer, - } -} - -func buildSourceReferences(rawSourceURLs []any) []sourceReference { - priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"} - prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}} - - prioritized := make(map[string]sourceReference) - fallback := make([]sourceReference, 0, len(rawSourceURLs)) - seen := make(map[string]struct{}) - - for _, source := range rawSourceURLs { - item, ok := source.(map[string]any) - if !ok { - continue - } - - sourceURL, _ := item["sourceUrl"].(string) - sourceName, _ := item["sourceName"].(string) - sourceURL = strings.TrimSpace(sourceURL) - sourceName = strings.TrimSpace(sourceName) - if sourceURL == "" { - continue - } - - if _, exists := seen[sourceURL]; exists { - continue - } - seen[sourceURL] = struct{}{} - - ref := sourceReference{URL: sourceURL, Name: sourceName} - normalized := strings.ToLower(sourceName) - if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider { - if _, exists := prioritized[normalized]; !exists { - prioritized[normalized] = ref - } - continue - } - - fallback = append(fallback, ref) - } - - ordered := make([]sourceReference, 0, len(prioritized)+len(fallback)) - for _, provider := range priorityOrder { - if ref, ok := prioritized[provider]; ok { - ordered = append(ordered, ref) - } - } - - ordered = append(ordered, fallback...) - return ordered -} - -func extractEpisodeData(data map[string]any) (map[string]any, error) { - episodeData, ok := data["episode"].(map[string]any) - if ok && episodeData != nil { - return episodeData, nil - } - - toBeParsed, ok := data["tobeparsed"].(string) - if !ok || strings.TrimSpace(toBeParsed) == "" { - return nil, fmt.Errorf("episode not found") - } - - decoded, err := decryptTobeparsed(toBeParsed) - if err != nil { - return nil, fmt.Errorf("decode episode payload: %w", err) - } - - var parsed map[string]any - if err := json.Unmarshal(decoded, &parsed); err != nil { - return nil, fmt.Errorf("parse decoded payload: %w", err) - } - - episodeData, ok = parsed["episode"].(map[string]any) - if !ok || episodeData == nil { - return nil, fmt.Errorf("decoded payload missing episode") - } - - return episodeData, nil -} - -func decodeSourceURL(encoded string) string { - if encoded == "" { - return "" - } - - encoded = strings.TrimPrefix(encoded, "--") - - substitutions := map[string]string{ - "79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E", - "7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J", - "73": "K", "74": "L", "75": "M", "76": "N", "77": "O", - "68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T", - "6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y", - "62": "Z", - "59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e", - "5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j", - "53": "k", "54": "l", "55": "m", "56": "n", "57": "o", - "48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t", - "4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y", - "42": "z", - "08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4", - "0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9", - "15": "-", "16": ".", "67": "_", "46": "~", "02": ":", - "17": "/", "07": "?", "1b": "#", "63": "[", "65": "]", - "78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(", - "11": ")", "12": "*", "13": "+", "14": ",", "03": ";", - "05": "=", "1d": "%", - } - - var result strings.Builder - for idx := 0; idx < len(encoded); { - if idx+2 <= len(encoded) { - pair := encoded[idx : idx+2] - if sub, ok := substitutions[pair]; ok { - result.WriteString(sub) - idx += 2 - continue - } - } - - result.WriteByte(encoded[idx]) - idx++ - } - - decoded := result.String() - if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") { - decoded = strings.Replace(decoded, "/clock", "/clock.json", 1) - } - - return decoded -} - -func detectStreamType(sourceURL string) string { - lower := strings.ToLower(sourceURL) - if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") { - return "m3u8" - } - - if strings.Contains(lower, ".mp4") { - return "mp4" - } - - return "unknown" -} - -func detectEmbedType(rawURL string) string { - lower := strings.ToLower(rawURL) - embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"} - for _, host := range embedHosts { - if strings.Contains(lower, host) { - return "embed" - } - } - - return "unknown" -} - -func getMapKeys(m map[string]any) []string { - keys := make([]string, 0, len(m)) - for k := range m { - keys = append(keys, k) - } - return keys -} -``` - -- [ ] **Step 2: Commit** - -```bash -git add api/playback/allanime_extractor.go -git commit -m "feat: add allanime_extractor.go with moved logic" -``` - -### Task 2: Update allanime_client.go - -**Files:** -- Modify: `api/playback/allanime_client.go` - -- [ ] **Step 1: Remove moved logic from allanime_client.go** -Remove: `extractSourceURLsFromData`, `buildStreamSource`, `sourceReference` struct, `buildSourceReferences`, `extractEpisodeData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, `getMapKeys`. - -- [ ] **Step 2: Verify build** - -Run: `go build ./api/playback/...` -Expected: PASS - -- [ ] **Step 3: Commit** - -```bash -git add api/playback/allanime_client.go -git commit -m "refactor: remove moved logic from allanime_client.go" -``` diff --git a/docs/superpowers/plans/2026-05-06-project-structure-cleanup.md b/docs/superpowers/plans/2026-05-06-project-structure-cleanup.md deleted file mode 100644 index c267c71..0000000 --- a/docs/superpowers/plans/2026-05-06-project-structure-cleanup.md +++ /dev/null @@ -1,150 +0,0 @@ -# Project Structure Cleanup Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Standardize the Go project structure, consolidate database packages, and remove redundant files to improve maintainability and idiomatic code patterns. - -**Architecture:** -1. Consolidate all database-related code into `internal/db` with the package name `db`. -2. Remove the `internal/db/sqlite` sub-package and move its contents to `internal/db`. -3. Update `sqlc.yaml` and regenerate code. -4. Clean up the `tmp/scripts/` directory. -5. Update all imports throughout the project. - -**Tech Stack:** Go, sqlc, Bun - ---- - -### Task 1: Clean up tmp/scripts - -**Files:** -- Delete: `tmp/scripts/*.go` - -- [ ] **Step 1: Delete redundant scripts** - -Run: `rm tmp/scripts/*.go` - -- [ ] **Step 2: Commit cleanup** - -```bash -git add tmp/scripts/ -git commit -m "chore: remove redundant scripts" -``` - -### Task 2: Standardize internal/db Package and Flatten Structure - -**Files:** -- Modify: `sqlc.yaml` -- Move: `internal/db/sqlite/sqlite.go` -> `internal/db/sqlite.go` -- Modify: `internal/db/*.go` (generated and manual) - -- [ ] **Step 1: Update sqlc configuration** - -Modify `sqlc.yaml`: -```yaml -version: "2" -sql: - - engine: "sqlite" - queries: "internal/db/queries.sql" - schema: "migrations/" - gen: - go: - package: "db" - out: "internal/db" - emit_json_tags: true - emit_prepared_queries: false - emit_interface: true - emit_exact_table_names: false -``` - -- [ ] **Step 2: Regenerate sqlc code** - -Run: `sqlc generate` - -- [ ] **Step 3: Move and update sqlite.go** - -Run: `mv internal/db/sqlite/sqlite.go internal/db/sqlite.go && rm -rf internal/db/sqlite` - -Modify `internal/db/sqlite.go`: -```go -package db - -import ( - "database/sql" - "fmt" - "os" - "path/filepath" - - _ "github.com/mattn/go-sqlite3" -) -// ... existing functions, update return types to use local db package if needed -``` - -- [ ] **Step 4: Update package names in manual files** - -Modify `internal/db/helpers.go` and `internal/db/migrate.go` to use `package db`. - -- [ ] **Step 5: Commit database structural changes** - -```bash -git add internal/db/ sqlc.yaml -git commit -m "refactor: consolidate db package and flatten structure" -``` - -### Task 3: Update Global Imports - -**Files:** -- Modify: All files importing `mal/internal/db` or `mal/internal/db/sqlite` - -- [ ] **Step 1: Replace imports and qualifiers** - -Use `sed` or `edit` to update all imports: -- `database "mal/internal/db"` -> `"mal/internal/db"` -- `"mal/internal/db"` -> `"mal/internal/db"` (ensure it's used as `db.`) -- `"mal/internal/db/sqlite"` -> `"mal/internal/db"` - -- [ ] **Step 2: Update code references** - -Replace `database.` and `sqlite.` with `db.` globally where applicable. - -- [ ] **Step 3: Verify build** - -Run: `go build ./...` -Expected: PASS - -- [ ] **Step 4: Commit import updates** - -```bash -git add . -git commit -m "refactor: update imports to use new db package" -``` - -### Task 4: Clean dist/ and Verify Build - -**Files:** -- Modify: `package.json` (if needed) -- Clean: `dist/` - -- [ ] **Step 1: Clean dist directory** - -Run: `rm -rf dist/*` - -- [ ] **Step 2: Run frontend build** - -Run: `bun run build:assets` - -- [ ] **Step 3: Verify dist structure** - -Check if `dist/static/static` still exists. If so, investigate `bun build` command in `package.json`. - -- [ ] **Step 4: Final verification** - -Run: `go build ./... && bun run typecheck` -Expected: PASS - -- [ ] **Step 5: Commit build fixes** - -```bash -git add . -git commit -m "build: clean dist and verify assets" -``` diff --git a/docs/superpowers/plans/2026-05-06-rate-limiter-refactor.md b/docs/superpowers/plans/2026-05-06-rate-limiter-refactor.md deleted file mode 100644 index 1aa2cec..0000000 --- a/docs/superpowers/plans/2026-05-06-rate-limiter-refactor.md +++ /dev/null @@ -1,277 +0,0 @@ -# Rate Limiter Refactor Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Refactor the global-state rate limiter into a struct-based implementation that supports multiple instances with different configurations. - -**Architecture:** -1. Introduce a `Limiter` struct that encapsulates state and configuration. -2. Remove global variables and `init()` in `pkg/middleware/ratelimit.go`. -3. Provide generic and auth-specific middleware methods. -4. Integrate the new `Limiter` into the server startup and router. - -**Tech Stack:** Go (Standard Library) - ---- - -### Task 1: Refactor Limiter Logic and State - -**Files:** -- Modify: `pkg/middleware/ratelimit.go` - -- [ ] **Step 1: Define new structs and remove global state** - -Replace content of `pkg/middleware/ratelimit.go`: -```go -package middleware - -import ( - "fmt" - "net/http" - "strings" - "sync" - "time" -) - -type visitor struct { - attempts int - lastSeen time.Time -} - -type Config struct { - MaxAttempts int - Window time.Duration -} - -type Limiter struct { - mu sync.Mutex - visitors map[string]*visitor - config Config -} - -func NewLimiter(cfg Config) *Limiter { - return &Limiter{ - visitors: make(map[string]*visitor), - config: cfg, - } -} - -func (l *Limiter) Cleanup(now time.Time) { - l.mu.Lock() - defer l.mu.Unlock() - for ip, v := range l.visitors { - if now.Sub(v.lastSeen) > l.config.Window*3 { - delete(l.visitors, ip) - } - } -} - -func getIP(r *http.Request) string { - if xff := r.Header.Get("X-Forwarded-For"); xff != "" { - ips := strings.Split(xff, ",") - return strings.TrimSpace(ips[0]) - } - if realIP := r.Header.Get("X-Real-IP"); realIP != "" { - return realIP - } - ip := r.RemoteAddr - if colonIdx := strings.LastIndex(ip, ":"); colonIdx != -1 { - ip = ip[:colonIdx] - } - return ip -} -``` - -- [ ] **Step 2: Implement Middleware methods** - -Add to `pkg/middleware/ratelimit.go`: -```go -func (l *Limiter) Middleware(next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if !l.allow(getIP(r)) { - http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests) - return - } - next.ServeHTTP(w, r) - }) -} - -func (l *Limiter) AuthMiddleware(next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if !l.allow(getIP(r)) { - if strings.HasPrefix(r.URL.Path, "/") { - http.Redirect(w, r, fmt.Sprintf("%s?error=rate_limited", r.URL.Path), http.StatusFound) - return - } - http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests) - return - } - next.ServeHTTP(w, r) - }) -} - -func (l *Limiter) allow(ip string) bool { - l.mu.Lock() - defer l.mu.Unlock() - - v, exists := l.visitors[ip] - if !exists { - l.visitors[ip] = &visitor{1, time.Now()} - return true - } - - if time.Since(v.lastSeen) > l.config.Window { - v.attempts = 1 - v.lastSeen = time.Now() - return true - } - - v.attempts++ - v.lastSeen = time.Now() - return v.attempts <= l.config.MaxAttempts -} -``` - -- [ ] **Step 3: Commit refactor** - -```bash -git add pkg/middleware/ratelimit.go -git commit -m "refactor: convert rate limiter to struct-based implementation" -``` - -### Task 2: Add Unit Tests for Limiter - -**Files:** -- Create: `pkg/middleware/ratelimit_test.go` - -- [ ] **Step 1: Write tests for Limiter logic** - -Create `pkg/middleware/ratelimit_test.go`: -```go -package middleware - -import ( - "net/http" - "net/http/httptest" - "testing" - "time" -) - -func testHandler() http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(http.StatusOK) - }) -} - -func TestLimiter(t *testing.T) { - cfg := Config{MaxAttempts: 2, Window: 100 * time.Millisecond} - l := NewLimiter(cfg) - handler := l.Middleware(testHandler()) - - // First attempt - req := httptest.NewRequest("GET", "/", nil) - req.RemoteAddr = "1.2.3.4:1234" - rr := httptest.NewRecorder() - handler.ServeHTTP(rr, req) - if rr.Code != http.StatusOK { - t.Errorf("expected 200, got %d", rr.Code) - } - - // Second attempt - rr = httptest.NewRecorder() - handler.ServeHTTP(rr, req) - if rr.Code != http.StatusOK { - t.Errorf("expected 200, got %d", rr.Code) - } - - // Third attempt (should fail) - rr = httptest.NewRecorder() - handler.ServeHTTP(rr, req) - if rr.Code != http.StatusTooManyRequests { - t.Errorf("expected 429, got %d", rr.Code) - } - - // Wait for window to expire - time.Sleep(150 * time.Millisecond) - - // Fourth attempt (should pass again) - rr = httptest.NewRecorder() - handler.ServeHTTP(rr, req) - if rr.Code != http.StatusOK { - t.Errorf("expected 200, got %d", rr.Code) - } -} -``` - -- [ ] **Step 2: Run tests** - -Run: `go test -v pkg/middleware/ratelimit.go pkg/middleware/ratelimit_test.go` -Expected: PASS - -- [ ] **Step 3: Commit tests** - -```bash -git add pkg/middleware/ratelimit_test.go -git commit -m "test: add unit tests for rate limiter" -``` - -### Task 3: Integrate Limiter into Server - -**Files:** -- Modify: `internal/server/routes.go` -- Modify: `cmd/server/main.go` - -- [ ] **Step 1: Update Config and NewRouter** - -Modify `internal/server/routes.go`: -```go -type Config struct { - DB *db.Queries - SQLDB *sql.DB - JikanClient *jikan.Client - AuthService *auth.Service - PlaybackProxySecret string - AuthLimiter *pkgmiddleware.Limiter // Add this -} - -// ... in NewRouter function, replace: -// pkgmiddleware.RateLimitAuth(...) -// with: -// cfg.AuthLimiter.AuthMiddleware(...) -``` - -- [ ] **Step 2: Instantiate Limiter in main.go** - -Modify `cmd/server/main.go`: -```go -// ... in main function -authLimiter := pkgmiddleware.NewLimiter(pkgmiddleware.Config{ - MaxAttempts: 5, - Window: time.Minute, -}) - -// Start cleanup goroutine -go func() { - for { - time.Sleep(time.Minute) - authLimiter.Cleanup(time.Now()) - } -}() - -router := server.NewRouter(server.Config{ - // ... - AuthLimiter: authLimiter, -}) -``` - -- [ ] **Step 3: Verify build and manual test** - -Run: `go build ./...` -Expected: PASS - -- [ ] **Step 4: Commit integration** - -```bash -git add internal/server/routes.go cmd/server/main.go -git commit -m "feat: integrate new rate limiter into server" -``` diff --git a/docs/superpowers/plans/2026-05-06-refactor-import-watchlist.md b/docs/superpowers/plans/2026-05-06-refactor-import-watchlist.md deleted file mode 100644 index 4621b16..0000000 --- a/docs/superpowers/plans/2026-05-06-refactor-import-watchlist.md +++ /dev/null @@ -1,95 +0,0 @@ -# Refactor ImportWatchlist Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Refactor `ImportWatchlist` to use a database transaction for the import loop and improve logging. - -**Architecture:** Use `db.BeginTx` to wrap the loop. `ensureAnimeExists` continues to use `s.db` to avoid long-running transactions during external API calls. - -**Tech Stack:** Go, `database/sql`, `csv`. - ---- - -### Task 1: Refactor ImportWatchlist with Transaction - -**Files:** -- Modify: `api/watchlist/service.go` - -- [ ] **Step 1: Update ImportWatchlist implementation** - -```go -func (s *Service) ImportWatchlist(ctx context.Context, userID string, r io.Reader) error { - txQueries, tx, err := db.BeginTx(ctx, s.sqlDB) - if err != nil { - return fmt.Errorf("failed to begin transaction: %w", err) - } - defer tx.Rollback() - - reader := csv.NewReader(r) - if _, err := reader.Read(); err != nil { - return fmt.Errorf("failed to read csv header: %w", err) - } - - records, err := reader.ReadAll() - if err != nil { - return fmt.Errorf("failed to read csv records: %w", err) - } - - for i, record := range records { - if len(record) < 4 { - log.Printf("skipping row %d: insufficient columns", i+2) // i+2 because i is 0-indexed record after header - continue - } - - animeID, err := strconv.ParseInt(record[0], 10, 64) - if err != nil { - return fmt.Errorf("row %d: invalid anime id: %w", i+2, err) - } - - status := record[1] - if _, ok := validStatuses[status]; !ok { - status = "plan_to_watch" - } - - currentEpisode, _ := strconv.ParseInt(record[2], 10, 64) - currentTimeSeconds, _ := strconv.ParseFloat(record[3], 64) - - if err := s.ensureAnimeExists(ctx, animeID); err != nil { - return fmt.Errorf("row %d: failed to ensure anime: %w", i+2, err) - } - - _, err = txQueries.UpsertWatchListEntry(ctx, db.UpsertWatchListEntryParams{ - ID: uuid.New().String(), - UserID: userID, - AnimeID: animeID, - Status: status, - CurrentEpisode: sql.NullInt64{Int64: currentEpisode, Valid: currentEpisode > 0}, - CurrentTimeSeconds: currentTimeSeconds, - }) - if err != nil { - return fmt.Errorf("row %d: failed to upsert entry: %w", i+2, err) - } - } - - return tx.Commit() -} -``` - -- [ ] **Step 2: Add missing import `log`** - -- [ ] **Step 3: Verify compilation** - -Run: `go build ./api/watchlist/...` -Expected: Success (no output) - -- [ ] **Step 4: Verify tests** - -Run: `go test ./api/watchlist/...` -Expected: Success - -- [ ] **Step 5: Commit** - -```bash -git add api/watchlist/service.go -git commit -m "refactor: wrap watchlist import in transaction" -``` diff --git a/docs/superpowers/plans/2026-05-06-watchlist-export-import.md b/docs/superpowers/plans/2026-05-06-watchlist-export-import.md deleted file mode 100644 index fcb1e25..0000000 --- a/docs/superpowers/plans/2026-05-06-watchlist-export-import.md +++ /dev/null @@ -1,219 +0,0 @@ -# Watchlist CSV Export and Import Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add CSV export and import functionality to the anime watchlist to allow users to backup and restore their lists. - -**Architecture:** Client-side CSV generation for export using JavaScript Blobs. Server-side CSV parsing for import using a new API endpoint and Go's `encoding/csv` package. - -**Tech Stack:** Go (Backend), HTML/JavaScript (Frontend), Tailwind CSS (Styling). - ---- - -### Task 1: Add Import Logic to Watchlist Service - -**Files:** -- Modify: `api/watchlist/service.go` - -- [ ] **Step 1: Add ImportWatchlist method** -Add a method to handle CSV parsing and entry upsertion. - -```go -func (s *Service) ImportWatchlist(ctx context.Context, userID string, r io.Reader) error { - reader := csv.NewReader(r) - // Read header - if _, err := reader.Read(); err != nil { - return fmt.Errorf("failed to read csv header: %w", err) - } - - records, err := reader.ReadAll() - if err != nil { - return fmt.Errorf("failed to read csv records: %w", err) - } - - for i, record := range records { - if len(record) < 4 { - continue // Skip malformed rows - } - - animeID, err := strconv.ParseInt(record[0], 10, 64) - if err != nil { - return fmt.Errorf("row %d: invalid anime id: %w", i+1, err) - } - - status := record[1] - if _, ok := validStatuses[status]; !ok { - status = "plan_to_watch" - } - - currentEpisode, _ := strconv.ParseInt(record[2], 10, 64) - currentTimeSeconds, _ := strconv.ParseFloat(record[3], 64) - - if err := s.ensureAnimeExists(ctx, animeID); err != nil { - return fmt.Errorf("row %d: failed to ensure anime: %w", i+1, err) - } - - _, err = s.db.UpsertWatchListEntry(ctx, db.UpsertWatchListEntryParams{ - ID: uuid.New().String(), - UserID: userID, - AnimeID: animeID, - Status: status, - CurrentEpisode: sql.NullInt64{Int64: currentEpisode, Valid: currentEpisode > 0}, - CurrentTimeSeconds: currentTimeSeconds, - }) - if err != nil { - return fmt.Errorf("row %d: failed to upsert entry: %w", i+1, err) - } - } - - return nil -} -``` - -- [ ] **Step 2: Commit** - -```bash -git add api/watchlist/service.go -git commit -m "feat: add ImportWatchlist to service" -``` - -### Task 2: Create Import API Endpoint - -**Files:** -- Modify: `api/watchlist/handler.go` -- Modify: `internal/server/routes.go` - -- [ ] **Step 1: Add HandleImportWatchlist to Handler** - -```go -func (h *Handler) HandleImportWatchlist(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - http.Error(w, "method not allowed", http.StatusMethodNotAllowed) - return - } - - user := middleware.GetUser(r.Context()) - if user == nil { - http.Error(w, "unauthorized", http.StatusUnauthorized) - return - } - - file, _, err := r.FormFile("file") - if err != nil { - http.Error(w, "failed to get file from request", http.StatusBadRequest) - return - } - defer file.Close() - - if err := h.service.ImportWatchlist(r.Context(), user.ID, file); err != nil { - log.Printf("import failed: %v", err) - http.Error(w, "import failed: "+err.Error(), http.StatusInternalServerError) - return - } - - w.Header().Set("HX-Redirect", "/watchlist") - w.WriteHeader(http.StatusOK) -} -``` - -- [ ] **Step 2: Register route in internal/server/routes.go** - -```go -// Find existing watchlist routes and add: -mux.HandleFunc("/api/watchlist/import", watchlistHandler.HandleImportWatchlist) -``` - -- [ ] **Step 3: Commit** - -```bash -git add api/watchlist/handler.go internal/server/routes.go -git commit -m "feat: add watchlist import api endpoint" -``` - -### Task 3: Update Frontend Templates with Export/Import UI - -**Files:** -- Modify: `templates/watchlist.gohtml` -- Modify: `templates/watchlist_partial.gohtml` - -- [ ] **Step 1: Add buttons and JS logic to templates** -Add the UI components and scripts for CSV generation and file upload. - -```html - -
-
- -
- - -
-
- - - -``` - -- [ ] **Step 2: Commit** - -```bash -git add templates/watchlist.gohtml templates/watchlist_partial.gohtml -git commit -m "feat: add export/import buttons and logic to watchlist UI" -``` diff --git a/docs/superpowers/plans/2026-05-06-watchlist-import-transaction.md b/docs/superpowers/plans/2026-05-06-watchlist-import-transaction.md deleted file mode 100644 index f798415..0000000 --- a/docs/superpowers/plans/2026-05-06-watchlist-import-transaction.md +++ /dev/null @@ -1,20 +0,0 @@ -# Watchlist Import Improvement Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Wrap the watchlist import loop in a transaction and log skipped rows. - -**Architecture:** Use `db.BeginTx` to start a transaction and ensure all operations within the loop use the transaction-aware `txQueries`. - -**Tech Stack:** Go, standard library `database/sql` and `log`. - ---- - -### Task 1: Refactor ImportWatchlist to use Transaction - -**Files:** -- Modify: `/Users/mkelvers/dev/personal/mal/api/watchlist/service.go` - -- [ ] **Step 1: Apply transaction wrapper and logging** - -- [ ] **Step 2: Commit changes** diff --git a/docs/superpowers/specs/2026-05-06-allanime-client-decomposition-design.md b/docs/superpowers/specs/2026-05-06-allanime-client-decomposition-design.md deleted file mode 100644 index 3fb8d5d..0000000 --- a/docs/superpowers/specs/2026-05-06-allanime-client-decomposition-design.md +++ /dev/null @@ -1,41 +0,0 @@ -# Design Spec: AllAnime Client Decomposition - -Decompose the monolithic `api/playback/allanime_client.go` into focused, maintainable units to improve readability, testability, and separation of concerns. - -## 1. File Structure & Responsibilities - -The current 1200-line file will be split into the following components: - -### `pkg/net/utls/utls.go` (New) -- **Responsibility:** Generic uTLS and HTTP2 networking utilities. -- **Contents:** `utlsRoundTripper` struct and its `RoundTrip` implementation. -- **Benefit:** Decouples complex networking logic from domain-specific client code. - -### `api/playback/allanime_crypto.go` (New) -- **Responsibility:** AES decryption and dynamic key discovery. -- **Contents:** `decryptTobeparsed`, `getAESKey`, `fetchKeyFromForks`, and supporting validation/decryption logic. -- **Benefit:** Isolates the "hacky" key-scraping and crypto logic from the clean API client. - -### `api/playback/allanime_extractor.go` (New) -- **Responsibility:** Parsing raw data and detecting media types. -- **Contents:** `extractSourceURLsFromData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, and source-building helpers. -- **Benefit:** Clean separation of data transformation logic. - -### `api/playback/allanime_client.go` (Modified) -- **Responsibility:** Main service interface and GraphQL communication. -- **Contents:** `Search`, `GetEpisodes`, `GetEpisodeSources`, and `graphqlRequest`. -- **Benefit:** The file becomes a thin wrapper around the API protocol. - -## 2. Interface Stability -- The existing `allAnimeClient` struct and its public methods (`Search`, `GetEpisodes`, etc.) will maintain their signatures. -- Internal helper functions will be moved to the new files but remain within the `playback` package (except for the generic networking utility). - -## 3. Verification Plan - -### Automated Checks -- `go test ./api/playback/...`: Ensure all existing tests pass after the split. -- `go build ./...`: Verify no compilation errors due to visibility or package changes. -- Add new unit tests in `api/playback/allanime_crypto_test.go` for the moved decryption logic. - -### Manual Checks -- Verify that anime playback and searching still function correctly in the application. diff --git a/docs/superpowers/specs/2026-05-06-project-structure-cleanup-design.md b/docs/superpowers/specs/2026-05-06-project-structure-cleanup-design.md deleted file mode 100644 index 22a6ad8..0000000 --- a/docs/superpowers/specs/2026-05-06-project-structure-cleanup-design.md +++ /dev/null @@ -1,38 +0,0 @@ -# Design Spec: Project Structure Cleanup and Standardization - -Standardize the Go project structure, consolidate database packages, and remove redundant files to improve maintainability and idiomatic code patterns. - -## 1. Database Package Consolidation - -### Current State -- `internal/db` contains `sqlc` generated code in package `database`. -- `internal/db/sqlite/sqlite.go` is in a separate package for DB initialization. -- Imports are scattered between `mal/internal/db` (often aliased) and `mal/internal/db/sqlite`. - -### Changes -- **Package Renaming:** Change package `database` to `db` in all `internal/db/*.go` files. -- **sqlc configuration:** Update `sqlc.yaml` to use `package: db`. -- **Flattening:** Move `internal/db/sqlite/sqlite.go` to `internal/db/sqlite.go`. -- **Import Cleanup:** Update all consumers to use `mal/internal/db` directly. - -## 2. Redundant File Removal - -### Changes -- Delete all `.go` files in `tmp/scripts/` (`fix_json.go`, `fix_proxy.go`, `test_jikan.go`, etc.). -- Remove the empty directory `internal/db/sqlite` after moving the file. - -## 3. Asset Build Optimization - -### Changes -- Investigate `dist/` duplication. -- Update `package.json` build scripts if necessary to ensure `dist/` has a flat, clean structure matching what `NewRouter` expects. - -## 4. Verification Plan - -### Automated Checks -- `go build ./...`: Verify all Go imports and package renames. -- `bun run typecheck`: Verify TS/JS integrity. -- `sqlc generate`: Verify that regeneration doesn't break the new package naming. - -### Manual Checks -- Verify server starts and database migrations apply correctly. diff --git a/docs/superpowers/specs/2026-05-06-rate-limiter-refactor-design.md b/docs/superpowers/specs/2026-05-06-rate-limiter-refactor-design.md deleted file mode 100644 index 05965e3..0000000 --- a/docs/superpowers/specs/2026-05-06-rate-limiter-refactor-design.md +++ /dev/null @@ -1,68 +0,0 @@ -# Design Spec: Generic Rate Limiter Refactor - -Refactor the current global-state rate limiter into a struct-based implementation that supports multiple instances with different configurations, improving testability and flexibility. - -## 1. Core Architecture - -### Current State -- `pkg/middleware/ratelimit.go` uses global variables (`visitors`, `mu`, `quit`). -- Cleanup logic is started automatically via `init()`. -- Limits are hardcoded (5 attempts per minute). - -### Changes -- **Limiter Struct:** Encapsulate state within a `Limiter` struct. -- **Explicit Configuration:** Pass limits and windows via a `Config` struct. -- **Context-aware Cleanup:** Use `context.Context` to manage the cleanup goroutine lifecycle instead of a global `quit` channel. - -## 2. Component Design - -### `Limiter` Struct -```go -type visitor struct { - attempts int - lastSeen time.Time -} - -type Config struct { - MaxAttempts int - Window time.Duration -} - -type Limiter struct { - mu sync.Mutex - visitors map[string]*visitor - config Config -} -``` - -### Initialization -```go -func NewLimiter(cfg Config) *Limiter { - return &Limiter{ - visitors: make(map[string]*visitor), - config: cfg, - } -} -``` - -### Middleware Methods -- `Middleware(next http.Handler) http.Handler`: Generic rate limiting. -- `AuthMiddleware(next http.Handler) http.Handler`: Specific logic for auth (redirects with error params). - -## 3. Integration Plan - -- Instantiate the `Limiter` in `cmd/server/main.go`. -- Pass the `Limiter` or its middleware to `internal/server/NewRouter`. -- Update `internal/server/routes.go` to use the new instance-based middleware. - -## 4. Verification Plan - -### Automated Checks -- **Unit Tests:** Create `pkg/middleware/ratelimit_test.go` to verify: - - Multiple IPs are tracked independently. - - Limits are enforced correctly. - - Attempts reset after the window expires. -- `go test ./pkg/middleware/...`: Run the new tests. - -### Manual Checks -- Verify that exceeding login attempts still triggers the `rate_limited` redirect. diff --git a/docs/superpowers/specs/2026-05-06-watchlist-export-import-design.md b/docs/superpowers/specs/2026-05-06-watchlist-export-import-design.md deleted file mode 100644 index 1c1f337..0000000 --- a/docs/superpowers/specs/2026-05-06-watchlist-export-import-design.md +++ /dev/null @@ -1,50 +0,0 @@ -# Spec: Watchlist CSV Export and Import - -## Overview -Add functionality to export the user's watchlist to a CSV file and import it back. This allows for easy backup, external editing, and migration of watchlist data. - -## Requirements -- **Export**: Generate a CSV file containing `anime_id`, `status`, `current_episode`, and `current_time_seconds`. -- **Import**: Upload a CSV file to update or add watchlist entries. -- **Data Integrity**: - - Do not export `created_at` or `updated_at`. - - On import, overwrite existing entries with the same `anime_id`. - - Ensure anime exists in the local database before adding to watchlist (fetch from Jikan if necessary). -- **UI**: Add Export and Import buttons to the watchlist page header. - -## Architecture - -### Frontend -- **Template**: Update `templates/watchlist.gohtml` (and partial) to include Export/Import buttons. -- **Export Implementation**: - - Use JavaScript to iterate over the `AllEntries` data passed to the template. - - Create a CSV string. - - Trigger a browser download using a `Blob` and a temporary link element. -- **Import Implementation**: - - Use a hidden ``. - - When a file is selected, use `FormData` to upload it to the server via `fetch`. - - Show a simple alert or refresh the page on success. - -### Backend -- **Endpoint**: `POST /api/watchlist/import` -- **Service**: Add `ImportWatchlist(ctx, userID, reader)` to `api/watchlist/service.go`. -- **Logic**: - - Parse CSV using `encoding/csv`. - - For each record: - - Validate data types. - - Call `ensureAnimeExists` (existing service method). - - Use `UpsertWatchListEntry` to save/update. - - Process the entire file in a transaction if possible, or gracefully skip malformed rows. - -## Data Format (CSV) -Headers: `anime_id,status,current_episode,current_time_seconds` -Example: -```csv -123,watching,5,450.5 -456,completed,12,0 -``` - -## Success Criteria -1. Clicking "Export" downloads a valid CSV file. -2. Modifying the CSV and clicking "Import" correctly updates the watchlist in the UI and database. -3. New anime added via CSV are correctly fetched from Jikan and added to the database.