fuck superpowers
This commit is contained in:
@@ -1,167 +0,0 @@
|
||||
# AllAnime Client Decomposition Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Decompose the monolithic `api/playback/allanime_client.go` into focused, maintainable units.
|
||||
|
||||
**Architecture:**
|
||||
1. Move uTLS networking logic to `pkg/net/utls`.
|
||||
2. Move crypto and key discovery to `api/playback/allanime_crypto.go`.
|
||||
3. Move data extraction and stream detection to `api/playback/allanime_extractor.go`.
|
||||
4. Keep core API and GraphQL logic in `api/playback/allanime_client.go`.
|
||||
|
||||
**Tech Stack:** Go (uTLS, SHA256, AES-CTR)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Move uTLS RoundTripper to pkg/net/utls
|
||||
|
||||
**Files:**
|
||||
- Create: `pkg/net/utls/utls.go`
|
||||
- Modify: `api/playback/allanime_client.go`
|
||||
|
||||
- [ ] **Step 1: Create pkg/net/utls/utls.go**
|
||||
|
||||
```go
|
||||
package utls
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
utls "github.com/refraction-networking/utls"
|
||||
"golang.org/x/net/http2"
|
||||
)
|
||||
|
||||
type UtlsRoundTripper struct{}
|
||||
|
||||
func (rt *UtlsRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
port := req.URL.Port()
|
||||
if port == "" {
|
||||
if req.URL.Scheme == "https" {
|
||||
port = "443"
|
||||
} else {
|
||||
port = "80"
|
||||
}
|
||||
}
|
||||
|
||||
addr := net.JoinHostPort(req.URL.Hostname(), port)
|
||||
conn, err := net.DialTimeout("tcp", addr, 10*time.Second)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
config := &utls.Config{ServerName: req.URL.Hostname()}
|
||||
uConn := utls.UClient(conn, config, utls.HelloFirefox_Auto)
|
||||
|
||||
if err := uConn.Handshake(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var httpRT http.RoundTripper
|
||||
switch uConn.ConnectionState().NegotiatedProtocol {
|
||||
case http2.NextProtoTLS:
|
||||
t2 := &http2.Transport{
|
||||
DialTLSContext: func(ctx context.Context, network, addr string, cfg *utls.Config) (net.Conn, error) {
|
||||
return uConn, nil
|
||||
},
|
||||
}
|
||||
httpRT = t2
|
||||
default:
|
||||
t1 := &http.Transport{
|
||||
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
return uConn, nil
|
||||
},
|
||||
}
|
||||
httpRT = t1
|
||||
}
|
||||
|
||||
return httpRT.RoundTrip(req)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update allanime_client.go to use the new package**
|
||||
|
||||
Remove `utlsRoundTripper` from `api/playback/allanime_client.go` and update `newAllAnimeClient` to use `&utls.UtlsRoundTripper{}`.
|
||||
|
||||
- [ ] **Step 3: Verify build and commit**
|
||||
|
||||
Run: `go build ./api/playback/...`
|
||||
Expected: PASS
|
||||
|
||||
```bash
|
||||
git add pkg/net/utls/utls.go api/playback/allanime_client.go
|
||||
git commit -m "refactor: move utls roundtripper to pkg/net/utls"
|
||||
```
|
||||
|
||||
### Task 2: Extract Crypto and Key Discovery
|
||||
|
||||
**Files:**
|
||||
- Create: `api/playback/allanime_crypto.go`
|
||||
- Modify: `api/playback/allanime_client.go`
|
||||
|
||||
- [ ] **Step 1: Move crypto logic to allanime_crypto.go**
|
||||
|
||||
Extract constants (`allAnimeAESKey`, `aniCliRawSourceURL`, etc.) and functions (`decryptTobeparsed`, `tryDecryptCTR`, `getAESKey`, `fetchKeyFromForks`, `extractKey`, `getTestPayload`, `validateKeys`, `getAllKeys`) to `api/playback/allanime_crypto.go`.
|
||||
|
||||
- [ ] **Step 2: Update allanime_client.go**
|
||||
|
||||
Remove the extracted code from `api/playback/allanime_client.go`.
|
||||
|
||||
- [ ] **Step 3: Verify build and commit**
|
||||
|
||||
Run: `go build ./api/playback/...`
|
||||
Expected: PASS
|
||||
|
||||
```bash
|
||||
git add api/playback/allanime_crypto.go api/playback/allanime_client.go
|
||||
git commit -m "refactor: extract allanime crypto and key discovery"
|
||||
```
|
||||
|
||||
### Task 3: Extract Data Extraction and Stream Detection
|
||||
|
||||
**Files:**
|
||||
- Create: `api/playback/allanime_extractor.go`
|
||||
- Modify: `api/playback/allanime_client.go`
|
||||
|
||||
- [ ] **Step 1: Move extraction logic to allanime_extractor.go**
|
||||
|
||||
Extract functions (`extractSourceURLsFromData`, `buildStreamSource`, `sourceReference` struct, `buildSourceReferences`, `extractEpisodeData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, `getMapKeys`) to `api/playback/allanime_extractor.go`.
|
||||
|
||||
- [ ] **Step 2: Update allanime_client.go**
|
||||
|
||||
Remove the extracted code from `api/playback/allanime_client.go`. Ensure `allAnimeClient` methods still call these (now package-level) functions.
|
||||
|
||||
- [ ] **Step 3: Verify build and commit**
|
||||
|
||||
Run: `go build ./api/playback/...`
|
||||
Expected: PASS
|
||||
|
||||
```bash
|
||||
git add api/playback/allanime_extractor.go api/playback/allanime_client.go
|
||||
git commit -m "refactor: extract allanime data extraction logic"
|
||||
```
|
||||
|
||||
### Task 4: Final Cleanup and Verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `api/playback/allanime_client.go`
|
||||
|
||||
- [ ] **Step 1: Final review of allanime_client.go**
|
||||
|
||||
Ensure only `allAnimeClient` struct, `newAllAnimeClient`, `graphqlRequest`, `graphqlRequestWithHash`, `Search`, `GetEpisodes`, `GetAvailableEpisodes`, and `GetEpisodeMetadata` remain.
|
||||
|
||||
- [ ] **Step 2: Run all tests**
|
||||
|
||||
Run: `go test -v ./api/playback/...`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 3: Commit final cleanup**
|
||||
|
||||
```bash
|
||||
git add api/playback/allanime_client.go
|
||||
git commit -m "refactor: final cleanup of allanime_client.go"
|
||||
```
|
||||
@@ -1,300 +0,0 @@
|
||||
# Extract AllAnime Data Extraction Logic Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extract data parsing and stream detection logic from `allanime_client.go` into a new file `allanime_extractor.go` to improve maintainability and separate concerns.
|
||||
|
||||
**Architecture:** Move non-network utility functions and helper structs related to parsing AllAnime responses and URL decoding/detection into a dedicated extractor file within the same package.
|
||||
|
||||
**Tech Stack:** Go (Standard Library)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Create allanime_extractor.go
|
||||
|
||||
**Files:**
|
||||
- Create: `api/playback/allanime_extractor.go`
|
||||
|
||||
- [ ] **Step 1: Create the file with moved logic**
|
||||
|
||||
```go
|
||||
package playback
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type sourceReference struct {
|
||||
URL string
|
||||
Name string
|
||||
}
|
||||
|
||||
func (c *allAnimeClient) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource {
|
||||
episodeData, ok := data["episode"].(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
|
||||
sourceURLs, ok := episodeData["sourceUrls"].([]any)
|
||||
if !ok || len(sourceURLs) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
references := buildSourceReferences(sourceURLs)
|
||||
if len(references) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
out := make([]StreamSource, 0, len(references))
|
||||
for _, ref := range references {
|
||||
target := strings.TrimSpace(ref.URL)
|
||||
if target == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") {
|
||||
sourceType := detectStreamType(target)
|
||||
if sourceType == "unknown" {
|
||||
sourceType = detectEmbedType(target)
|
||||
}
|
||||
|
||||
out = append(out, buildStreamSource(target, sourceType, ref.Name))
|
||||
continue
|
||||
}
|
||||
|
||||
decoded := decodeSourceURL(target)
|
||||
if decoded == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if strings.HasPrefix(decoded, "http://") || strings.HasPrefix(decoded, "https://") {
|
||||
sourceType := detectStreamType(decoded)
|
||||
if sourceType == "unknown" {
|
||||
sourceType = detectEmbedType(decoded)
|
||||
}
|
||||
|
||||
out = append(out, buildStreamSource(decoded, sourceType, ref.Name))
|
||||
continue
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(decoded, "/") {
|
||||
decoded = "/" + decoded
|
||||
}
|
||||
|
||||
extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded)
|
||||
if err != nil {
|
||||
log.Printf("source extraction failed for %s: %v", decoded, err)
|
||||
continue
|
||||
}
|
||||
|
||||
out = append(out, extracted...)
|
||||
}
|
||||
|
||||
return out
|
||||
}
|
||||
|
||||
func buildStreamSource(url, sourceType, provider string) StreamSource {
|
||||
return StreamSource{
|
||||
URL: url,
|
||||
Provider: provider,
|
||||
Type: sourceType,
|
||||
Referer: allAnimeReferer,
|
||||
}
|
||||
}
|
||||
|
||||
func buildSourceReferences(rawSourceURLs []any) []sourceReference {
|
||||
priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"}
|
||||
prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}}
|
||||
|
||||
prioritized := make(map[string]sourceReference)
|
||||
fallback := make([]sourceReference, 0, len(rawSourceURLs))
|
||||
seen := make(map[string]struct{})
|
||||
|
||||
for _, source := range rawSourceURLs {
|
||||
item, ok := source.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
|
||||
sourceURL, _ := item["sourceUrl"].(string)
|
||||
sourceName, _ := item["sourceName"].(string)
|
||||
sourceURL = strings.TrimSpace(sourceURL)
|
||||
sourceName = strings.TrimSpace(sourceName)
|
||||
if sourceURL == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if _, exists := seen[sourceURL]; exists {
|
||||
continue
|
||||
}
|
||||
seen[sourceURL] = struct{}{}
|
||||
|
||||
ref := sourceReference{URL: sourceURL, Name: sourceName}
|
||||
normalized := strings.ToLower(sourceName)
|
||||
if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider {
|
||||
if _, exists := prioritized[normalized]; !exists {
|
||||
prioritized[normalized] = ref
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
fallback = append(fallback, ref)
|
||||
}
|
||||
|
||||
ordered := make([]sourceReference, 0, len(prioritized)+len(fallback))
|
||||
for _, provider := range priorityOrder {
|
||||
if ref, ok := prioritized[provider]; ok {
|
||||
ordered = append(ordered, ref)
|
||||
}
|
||||
}
|
||||
|
||||
ordered = append(ordered, fallback...)
|
||||
return ordered
|
||||
}
|
||||
|
||||
func extractEpisodeData(data map[string]any) (map[string]any, error) {
|
||||
episodeData, ok := data["episode"].(map[string]any)
|
||||
if ok && episodeData != nil {
|
||||
return episodeData, nil
|
||||
}
|
||||
|
||||
toBeParsed, ok := data["tobeparsed"].(string)
|
||||
if !ok || strings.TrimSpace(toBeParsed) == "" {
|
||||
return nil, fmt.Errorf("episode not found")
|
||||
}
|
||||
|
||||
decoded, err := decryptTobeparsed(toBeParsed)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode episode payload: %w", err)
|
||||
}
|
||||
|
||||
var parsed map[string]any
|
||||
if err := json.Unmarshal(decoded, &parsed); err != nil {
|
||||
return nil, fmt.Errorf("parse decoded payload: %w", err)
|
||||
}
|
||||
|
||||
episodeData, ok = parsed["episode"].(map[string]any)
|
||||
if !ok || episodeData == nil {
|
||||
return nil, fmt.Errorf("decoded payload missing episode")
|
||||
}
|
||||
|
||||
return episodeData, nil
|
||||
}
|
||||
|
||||
func decodeSourceURL(encoded string) string {
|
||||
if encoded == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
encoded = strings.TrimPrefix(encoded, "--")
|
||||
|
||||
substitutions := map[string]string{
|
||||
"79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E",
|
||||
"7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J",
|
||||
"73": "K", "74": "L", "75": "M", "76": "N", "77": "O",
|
||||
"68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T",
|
||||
"6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y",
|
||||
"62": "Z",
|
||||
"59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e",
|
||||
"5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j",
|
||||
"53": "k", "54": "l", "55": "m", "56": "n", "57": "o",
|
||||
"48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t",
|
||||
"4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y",
|
||||
"42": "z",
|
||||
"08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4",
|
||||
"0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9",
|
||||
"15": "-", "16": ".", "67": "_", "46": "~", "02": ":",
|
||||
"17": "/", "07": "?", "1b": "#", "63": "[", "65": "]",
|
||||
"78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(",
|
||||
"11": ")", "12": "*", "13": "+", "14": ",", "03": ";",
|
||||
"05": "=", "1d": "%",
|
||||
}
|
||||
|
||||
var result strings.Builder
|
||||
for idx := 0; idx < len(encoded); {
|
||||
if idx+2 <= len(encoded) {
|
||||
pair := encoded[idx : idx+2]
|
||||
if sub, ok := substitutions[pair]; ok {
|
||||
result.WriteString(sub)
|
||||
idx += 2
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
result.WriteByte(encoded[idx])
|
||||
idx++
|
||||
}
|
||||
|
||||
decoded := result.String()
|
||||
if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") {
|
||||
decoded = strings.Replace(decoded, "/clock", "/clock.json", 1)
|
||||
}
|
||||
|
||||
return decoded
|
||||
}
|
||||
|
||||
func detectStreamType(sourceURL string) string {
|
||||
lower := strings.ToLower(sourceURL)
|
||||
if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") {
|
||||
return "m3u8"
|
||||
}
|
||||
|
||||
if strings.Contains(lower, ".mp4") {
|
||||
return "mp4"
|
||||
}
|
||||
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
func detectEmbedType(rawURL string) string {
|
||||
lower := strings.ToLower(rawURL)
|
||||
embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"}
|
||||
for _, host := range embedHosts {
|
||||
if strings.Contains(lower, host) {
|
||||
return "embed"
|
||||
}
|
||||
}
|
||||
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
func getMapKeys(m map[string]any) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add api/playback/allanime_extractor.go
|
||||
git commit -m "feat: add allanime_extractor.go with moved logic"
|
||||
```
|
||||
|
||||
### Task 2: Update allanime_client.go
|
||||
|
||||
**Files:**
|
||||
- Modify: `api/playback/allanime_client.go`
|
||||
|
||||
- [ ] **Step 1: Remove moved logic from allanime_client.go**
|
||||
Remove: `extractSourceURLsFromData`, `buildStreamSource`, `sourceReference` struct, `buildSourceReferences`, `extractEpisodeData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, `getMapKeys`.
|
||||
|
||||
- [ ] **Step 2: Verify build**
|
||||
|
||||
Run: `go build ./api/playback/...`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add api/playback/allanime_client.go
|
||||
git commit -m "refactor: remove moved logic from allanime_client.go"
|
||||
```
|
||||
@@ -1,150 +0,0 @@
|
||||
# Project Structure Cleanup Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Standardize the Go project structure, consolidate database packages, and remove redundant files to improve maintainability and idiomatic code patterns.
|
||||
|
||||
**Architecture:**
|
||||
1. Consolidate all database-related code into `internal/db` with the package name `db`.
|
||||
2. Remove the `internal/db/sqlite` sub-package and move its contents to `internal/db`.
|
||||
3. Update `sqlc.yaml` and regenerate code.
|
||||
4. Clean up the `tmp/scripts/` directory.
|
||||
5. Update all imports throughout the project.
|
||||
|
||||
**Tech Stack:** Go, sqlc, Bun
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Clean up tmp/scripts
|
||||
|
||||
**Files:**
|
||||
- Delete: `tmp/scripts/*.go`
|
||||
|
||||
- [ ] **Step 1: Delete redundant scripts**
|
||||
|
||||
Run: `rm tmp/scripts/*.go`
|
||||
|
||||
- [ ] **Step 2: Commit cleanup**
|
||||
|
||||
```bash
|
||||
git add tmp/scripts/
|
||||
git commit -m "chore: remove redundant scripts"
|
||||
```
|
||||
|
||||
### Task 2: Standardize internal/db Package and Flatten Structure
|
||||
|
||||
**Files:**
|
||||
- Modify: `sqlc.yaml`
|
||||
- Move: `internal/db/sqlite/sqlite.go` -> `internal/db/sqlite.go`
|
||||
- Modify: `internal/db/*.go` (generated and manual)
|
||||
|
||||
- [ ] **Step 1: Update sqlc configuration**
|
||||
|
||||
Modify `sqlc.yaml`:
|
||||
```yaml
|
||||
version: "2"
|
||||
sql:
|
||||
- engine: "sqlite"
|
||||
queries: "internal/db/queries.sql"
|
||||
schema: "migrations/"
|
||||
gen:
|
||||
go:
|
||||
package: "db"
|
||||
out: "internal/db"
|
||||
emit_json_tags: true
|
||||
emit_prepared_queries: false
|
||||
emit_interface: true
|
||||
emit_exact_table_names: false
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Regenerate sqlc code**
|
||||
|
||||
Run: `sqlc generate`
|
||||
|
||||
- [ ] **Step 3: Move and update sqlite.go**
|
||||
|
||||
Run: `mv internal/db/sqlite/sqlite.go internal/db/sqlite.go && rm -rf internal/db/sqlite`
|
||||
|
||||
Modify `internal/db/sqlite.go`:
|
||||
```go
|
||||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
_ "github.com/mattn/go-sqlite3"
|
||||
)
|
||||
// ... existing functions, update return types to use local db package if needed
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update package names in manual files**
|
||||
|
||||
Modify `internal/db/helpers.go` and `internal/db/migrate.go` to use `package db`.
|
||||
|
||||
- [ ] **Step 5: Commit database structural changes**
|
||||
|
||||
```bash
|
||||
git add internal/db/ sqlc.yaml
|
||||
git commit -m "refactor: consolidate db package and flatten structure"
|
||||
```
|
||||
|
||||
### Task 3: Update Global Imports
|
||||
|
||||
**Files:**
|
||||
- Modify: All files importing `mal/internal/db` or `mal/internal/db/sqlite`
|
||||
|
||||
- [ ] **Step 1: Replace imports and qualifiers**
|
||||
|
||||
Use `sed` or `edit` to update all imports:
|
||||
- `database "mal/internal/db"` -> `"mal/internal/db"`
|
||||
- `"mal/internal/db"` -> `"mal/internal/db"` (ensure it's used as `db.`)
|
||||
- `"mal/internal/db/sqlite"` -> `"mal/internal/db"`
|
||||
|
||||
- [ ] **Step 2: Update code references**
|
||||
|
||||
Replace `database.` and `sqlite.` with `db.` globally where applicable.
|
||||
|
||||
- [ ] **Step 3: Verify build**
|
||||
|
||||
Run: `go build ./...`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit import updates**
|
||||
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "refactor: update imports to use new db package"
|
||||
```
|
||||
|
||||
### Task 4: Clean dist/ and Verify Build
|
||||
|
||||
**Files:**
|
||||
- Modify: `package.json` (if needed)
|
||||
- Clean: `dist/`
|
||||
|
||||
- [ ] **Step 1: Clean dist directory**
|
||||
|
||||
Run: `rm -rf dist/*`
|
||||
|
||||
- [ ] **Step 2: Run frontend build**
|
||||
|
||||
Run: `bun run build:assets`
|
||||
|
||||
- [ ] **Step 3: Verify dist structure**
|
||||
|
||||
Check if `dist/static/static` still exists. If so, investigate `bun build` command in `package.json`.
|
||||
|
||||
- [ ] **Step 4: Final verification**
|
||||
|
||||
Run: `go build ./... && bun run typecheck`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit build fixes**
|
||||
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "build: clean dist and verify assets"
|
||||
```
|
||||
@@ -1,277 +0,0 @@
|
||||
# Rate Limiter Refactor Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Refactor the global-state rate limiter into a struct-based implementation that supports multiple instances with different configurations.
|
||||
|
||||
**Architecture:**
|
||||
1. Introduce a `Limiter` struct that encapsulates state and configuration.
|
||||
2. Remove global variables and `init()` in `pkg/middleware/ratelimit.go`.
|
||||
3. Provide generic and auth-specific middleware methods.
|
||||
4. Integrate the new `Limiter` into the server startup and router.
|
||||
|
||||
**Tech Stack:** Go (Standard Library)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Refactor Limiter Logic and State
|
||||
|
||||
**Files:**
|
||||
- Modify: `pkg/middleware/ratelimit.go`
|
||||
|
||||
- [ ] **Step 1: Define new structs and remove global state**
|
||||
|
||||
Replace content of `pkg/middleware/ratelimit.go`:
|
||||
```go
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type visitor struct {
|
||||
attempts int
|
||||
lastSeen time.Time
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
MaxAttempts int
|
||||
Window time.Duration
|
||||
}
|
||||
|
||||
type Limiter struct {
|
||||
mu sync.Mutex
|
||||
visitors map[string]*visitor
|
||||
config Config
|
||||
}
|
||||
|
||||
func NewLimiter(cfg Config) *Limiter {
|
||||
return &Limiter{
|
||||
visitors: make(map[string]*visitor),
|
||||
config: cfg,
|
||||
}
|
||||
}
|
||||
|
||||
func (l *Limiter) Cleanup(now time.Time) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
for ip, v := range l.visitors {
|
||||
if now.Sub(v.lastSeen) > l.config.Window*3 {
|
||||
delete(l.visitors, ip)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func getIP(r *http.Request) string {
|
||||
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
|
||||
ips := strings.Split(xff, ",")
|
||||
return strings.TrimSpace(ips[0])
|
||||
}
|
||||
if realIP := r.Header.Get("X-Real-IP"); realIP != "" {
|
||||
return realIP
|
||||
}
|
||||
ip := r.RemoteAddr
|
||||
if colonIdx := strings.LastIndex(ip, ":"); colonIdx != -1 {
|
||||
ip = ip[:colonIdx]
|
||||
}
|
||||
return ip
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement Middleware methods**
|
||||
|
||||
Add to `pkg/middleware/ratelimit.go`:
|
||||
```go
|
||||
func (l *Limiter) Middleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if !l.allow(getIP(r)) {
|
||||
http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests)
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
func (l *Limiter) AuthMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if !l.allow(getIP(r)) {
|
||||
if strings.HasPrefix(r.URL.Path, "/") {
|
||||
http.Redirect(w, r, fmt.Sprintf("%s?error=rate_limited", r.URL.Path), http.StatusFound)
|
||||
return
|
||||
}
|
||||
http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests)
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
func (l *Limiter) allow(ip string) bool {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
|
||||
v, exists := l.visitors[ip]
|
||||
if !exists {
|
||||
l.visitors[ip] = &visitor{1, time.Now()}
|
||||
return true
|
||||
}
|
||||
|
||||
if time.Since(v.lastSeen) > l.config.Window {
|
||||
v.attempts = 1
|
||||
v.lastSeen = time.Now()
|
||||
return true
|
||||
}
|
||||
|
||||
v.attempts++
|
||||
v.lastSeen = time.Now()
|
||||
return v.attempts <= l.config.MaxAttempts
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit refactor**
|
||||
|
||||
```bash
|
||||
git add pkg/middleware/ratelimit.go
|
||||
git commit -m "refactor: convert rate limiter to struct-based implementation"
|
||||
```
|
||||
|
||||
### Task 2: Add Unit Tests for Limiter
|
||||
|
||||
**Files:**
|
||||
- Create: `pkg/middleware/ratelimit_test.go`
|
||||
|
||||
- [ ] **Step 1: Write tests for Limiter logic**
|
||||
|
||||
Create `pkg/middleware/ratelimit_test.go`:
|
||||
```go
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func testHandler() http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
})
|
||||
}
|
||||
|
||||
func TestLimiter(t *testing.T) {
|
||||
cfg := Config{MaxAttempts: 2, Window: 100 * time.Millisecond}
|
||||
l := NewLimiter(cfg)
|
||||
handler := l.Middleware(testHandler())
|
||||
|
||||
// First attempt
|
||||
req := httptest.NewRequest("GET", "/", nil)
|
||||
req.RemoteAddr = "1.2.3.4:1234"
|
||||
rr := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rr, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("expected 200, got %d", rr.Code)
|
||||
}
|
||||
|
||||
// Second attempt
|
||||
rr = httptest.NewRecorder()
|
||||
handler.ServeHTTP(rr, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("expected 200, got %d", rr.Code)
|
||||
}
|
||||
|
||||
// Third attempt (should fail)
|
||||
rr = httptest.NewRecorder()
|
||||
handler.ServeHTTP(rr, req)
|
||||
if rr.Code != http.StatusTooManyRequests {
|
||||
t.Errorf("expected 429, got %d", rr.Code)
|
||||
}
|
||||
|
||||
// Wait for window to expire
|
||||
time.Sleep(150 * time.Millisecond)
|
||||
|
||||
// Fourth attempt (should pass again)
|
||||
rr = httptest.NewRecorder()
|
||||
handler.ServeHTTP(rr, req)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("expected 200, got %d", rr.Code)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests**
|
||||
|
||||
Run: `go test -v pkg/middleware/ratelimit.go pkg/middleware/ratelimit_test.go`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 3: Commit tests**
|
||||
|
||||
```bash
|
||||
git add pkg/middleware/ratelimit_test.go
|
||||
git commit -m "test: add unit tests for rate limiter"
|
||||
```
|
||||
|
||||
### Task 3: Integrate Limiter into Server
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/server/routes.go`
|
||||
- Modify: `cmd/server/main.go`
|
||||
|
||||
- [ ] **Step 1: Update Config and NewRouter**
|
||||
|
||||
Modify `internal/server/routes.go`:
|
||||
```go
|
||||
type Config struct {
|
||||
DB *db.Queries
|
||||
SQLDB *sql.DB
|
||||
JikanClient *jikan.Client
|
||||
AuthService *auth.Service
|
||||
PlaybackProxySecret string
|
||||
AuthLimiter *pkgmiddleware.Limiter // Add this
|
||||
}
|
||||
|
||||
// ... in NewRouter function, replace:
|
||||
// pkgmiddleware.RateLimitAuth(...)
|
||||
// with:
|
||||
// cfg.AuthLimiter.AuthMiddleware(...)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Instantiate Limiter in main.go**
|
||||
|
||||
Modify `cmd/server/main.go`:
|
||||
```go
|
||||
// ... in main function
|
||||
authLimiter := pkgmiddleware.NewLimiter(pkgmiddleware.Config{
|
||||
MaxAttempts: 5,
|
||||
Window: time.Minute,
|
||||
})
|
||||
|
||||
// Start cleanup goroutine
|
||||
go func() {
|
||||
for {
|
||||
time.Sleep(time.Minute)
|
||||
authLimiter.Cleanup(time.Now())
|
||||
}
|
||||
}()
|
||||
|
||||
router := server.NewRouter(server.Config{
|
||||
// ...
|
||||
AuthLimiter: authLimiter,
|
||||
})
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify build and manual test**
|
||||
|
||||
Run: `go build ./...`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit integration**
|
||||
|
||||
```bash
|
||||
git add internal/server/routes.go cmd/server/main.go
|
||||
git commit -m "feat: integrate new rate limiter into server"
|
||||
```
|
||||
@@ -1,95 +0,0 @@
|
||||
# Refactor ImportWatchlist Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Refactor `ImportWatchlist` to use a database transaction for the import loop and improve logging.
|
||||
|
||||
**Architecture:** Use `db.BeginTx` to wrap the loop. `ensureAnimeExists` continues to use `s.db` to avoid long-running transactions during external API calls.
|
||||
|
||||
**Tech Stack:** Go, `database/sql`, `csv`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Refactor ImportWatchlist with Transaction
|
||||
|
||||
**Files:**
|
||||
- Modify: `api/watchlist/service.go`
|
||||
|
||||
- [ ] **Step 1: Update ImportWatchlist implementation**
|
||||
|
||||
```go
|
||||
func (s *Service) ImportWatchlist(ctx context.Context, userID string, r io.Reader) error {
|
||||
txQueries, tx, err := db.BeginTx(ctx, s.sqlDB)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to begin transaction: %w", err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
reader := csv.NewReader(r)
|
||||
if _, err := reader.Read(); err != nil {
|
||||
return fmt.Errorf("failed to read csv header: %w", err)
|
||||
}
|
||||
|
||||
records, err := reader.ReadAll()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read csv records: %w", err)
|
||||
}
|
||||
|
||||
for i, record := range records {
|
||||
if len(record) < 4 {
|
||||
log.Printf("skipping row %d: insufficient columns", i+2) // i+2 because i is 0-indexed record after header
|
||||
continue
|
||||
}
|
||||
|
||||
animeID, err := strconv.ParseInt(record[0], 10, 64)
|
||||
if err != nil {
|
||||
return fmt.Errorf("row %d: invalid anime id: %w", i+2, err)
|
||||
}
|
||||
|
||||
status := record[1]
|
||||
if _, ok := validStatuses[status]; !ok {
|
||||
status = "plan_to_watch"
|
||||
}
|
||||
|
||||
currentEpisode, _ := strconv.ParseInt(record[2], 10, 64)
|
||||
currentTimeSeconds, _ := strconv.ParseFloat(record[3], 64)
|
||||
|
||||
if err := s.ensureAnimeExists(ctx, animeID); err != nil {
|
||||
return fmt.Errorf("row %d: failed to ensure anime: %w", i+2, err)
|
||||
}
|
||||
|
||||
_, err = txQueries.UpsertWatchListEntry(ctx, db.UpsertWatchListEntryParams{
|
||||
ID: uuid.New().String(),
|
||||
UserID: userID,
|
||||
AnimeID: animeID,
|
||||
Status: status,
|
||||
CurrentEpisode: sql.NullInt64{Int64: currentEpisode, Valid: currentEpisode > 0},
|
||||
CurrentTimeSeconds: currentTimeSeconds,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("row %d: failed to upsert entry: %w", i+2, err)
|
||||
}
|
||||
}
|
||||
|
||||
return tx.Commit()
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add missing import `log`**
|
||||
|
||||
- [ ] **Step 3: Verify compilation**
|
||||
|
||||
Run: `go build ./api/watchlist/...`
|
||||
Expected: Success (no output)
|
||||
|
||||
- [ ] **Step 4: Verify tests**
|
||||
|
||||
Run: `go test ./api/watchlist/...`
|
||||
Expected: Success
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add api/watchlist/service.go
|
||||
git commit -m "refactor: wrap watchlist import in transaction"
|
||||
```
|
||||
@@ -1,219 +0,0 @@
|
||||
# Watchlist CSV Export and Import Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add CSV export and import functionality to the anime watchlist to allow users to backup and restore their lists.
|
||||
|
||||
**Architecture:** Client-side CSV generation for export using JavaScript Blobs. Server-side CSV parsing for import using a new API endpoint and Go's `encoding/csv` package.
|
||||
|
||||
**Tech Stack:** Go (Backend), HTML/JavaScript (Frontend), Tailwind CSS (Styling).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Import Logic to Watchlist Service
|
||||
|
||||
**Files:**
|
||||
- Modify: `api/watchlist/service.go`
|
||||
|
||||
- [ ] **Step 1: Add ImportWatchlist method**
|
||||
Add a method to handle CSV parsing and entry upsertion.
|
||||
|
||||
```go
|
||||
func (s *Service) ImportWatchlist(ctx context.Context, userID string, r io.Reader) error {
|
||||
reader := csv.NewReader(r)
|
||||
// Read header
|
||||
if _, err := reader.Read(); err != nil {
|
||||
return fmt.Errorf("failed to read csv header: %w", err)
|
||||
}
|
||||
|
||||
records, err := reader.ReadAll()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read csv records: %w", err)
|
||||
}
|
||||
|
||||
for i, record := range records {
|
||||
if len(record) < 4 {
|
||||
continue // Skip malformed rows
|
||||
}
|
||||
|
||||
animeID, err := strconv.ParseInt(record[0], 10, 64)
|
||||
if err != nil {
|
||||
return fmt.Errorf("row %d: invalid anime id: %w", i+1, err)
|
||||
}
|
||||
|
||||
status := record[1]
|
||||
if _, ok := validStatuses[status]; !ok {
|
||||
status = "plan_to_watch"
|
||||
}
|
||||
|
||||
currentEpisode, _ := strconv.ParseInt(record[2], 10, 64)
|
||||
currentTimeSeconds, _ := strconv.ParseFloat(record[3], 64)
|
||||
|
||||
if err := s.ensureAnimeExists(ctx, animeID); err != nil {
|
||||
return fmt.Errorf("row %d: failed to ensure anime: %w", i+1, err)
|
||||
}
|
||||
|
||||
_, err = s.db.UpsertWatchListEntry(ctx, db.UpsertWatchListEntryParams{
|
||||
ID: uuid.New().String(),
|
||||
UserID: userID,
|
||||
AnimeID: animeID,
|
||||
Status: status,
|
||||
CurrentEpisode: sql.NullInt64{Int64: currentEpisode, Valid: currentEpisode > 0},
|
||||
CurrentTimeSeconds: currentTimeSeconds,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("row %d: failed to upsert entry: %w", i+1, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add api/watchlist/service.go
|
||||
git commit -m "feat: add ImportWatchlist to service"
|
||||
```
|
||||
|
||||
### Task 2: Create Import API Endpoint
|
||||
|
||||
**Files:**
|
||||
- Modify: `api/watchlist/handler.go`
|
||||
- Modify: `internal/server/routes.go`
|
||||
|
||||
- [ ] **Step 1: Add HandleImportWatchlist to Handler**
|
||||
|
||||
```go
|
||||
func (h *Handler) HandleImportWatchlist(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
user := middleware.GetUser(r.Context())
|
||||
if user == nil {
|
||||
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
file, _, err := r.FormFile("file")
|
||||
if err != nil {
|
||||
http.Error(w, "failed to get file from request", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
if err := h.service.ImportWatchlist(r.Context(), user.ID, file); err != nil {
|
||||
log.Printf("import failed: %v", err)
|
||||
http.Error(w, "import failed: "+err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("HX-Redirect", "/watchlist")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register route in internal/server/routes.go**
|
||||
|
||||
```go
|
||||
// Find existing watchlist routes and add:
|
||||
mux.HandleFunc("/api/watchlist/import", watchlistHandler.HandleImportWatchlist)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add api/watchlist/handler.go internal/server/routes.go
|
||||
git commit -m "feat: add watchlist import api endpoint"
|
||||
```
|
||||
|
||||
### Task 3: Update Frontend Templates with Export/Import UI
|
||||
|
||||
**Files:**
|
||||
- Modify: `templates/watchlist.gohtml`
|
||||
- Modify: `templates/watchlist_partial.gohtml`
|
||||
|
||||
- [ ] **Step 1: Add buttons and JS logic to templates**
|
||||
Add the UI components and scripts for CSV generation and file upload.
|
||||
|
||||
```html
|
||||
<!-- Buttons placement (in the header section) -->
|
||||
<div class="flex items-center gap-2">
|
||||
<div class="flex bg-white/5 p-1 rounded-md border border-white/10">
|
||||
<button type="button" class="px-3 py-1 text-xs hover:bg-white/10 transition-colors flex items-center gap-2 text-neutral-400 hover:text-white" onclick="exportWatchlistCSV()">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>
|
||||
Export
|
||||
</button>
|
||||
<div class="w-px h-4 bg-white/10 self-center"></div>
|
||||
<button type="button" class="px-3 py-1 text-xs hover:bg-white/10 transition-colors flex items-center gap-2 text-neutral-400 hover:text-white" onclick="document.getElementById('import-input').click()">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="17 8 12 3 7 8"/><line x1="12" y1="3" x2="12" y2="15"/></svg>
|
||||
Import
|
||||
</button>
|
||||
<input type="file" id="import-input" class="hidden" accept=".csv" onchange="importWatchlistCSV(this)">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- JS Logic -->
|
||||
<script>
|
||||
function exportWatchlistCSV() {
|
||||
const entries = {{json .AllEntries}};
|
||||
if (!entries || entries.length === 0) {
|
||||
alert('Watchlist is empty');
|
||||
return;
|
||||
}
|
||||
|
||||
let csv = 'anime_id,status,current_episode,current_time_seconds\n';
|
||||
entries.forEach(e => {
|
||||
csv += `${e.AnimeID},${e.Status},${e.CurrentEpisode.Int64},${e.CurrentTimeSeconds}\n`;
|
||||
});
|
||||
|
||||
const blob = new Blob([csv], { type: 'text/csv' });
|
||||
const url = window.URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.setAttribute('hidden', '');
|
||||
a.setAttribute('href', url);
|
||||
a.setAttribute('download', 'watchlist.csv');
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
}
|
||||
|
||||
async function importWatchlistCSV(input) {
|
||||
if (!input.files || input.files.length === 0) return;
|
||||
|
||||
const formData = new FormData();
|
||||
formData.append('file', input.files[0]);
|
||||
|
||||
try {
|
||||
const resp = await fetch('/api/watchlist/import', {
|
||||
method: 'POST',
|
||||
body: formData,
|
||||
headers: {
|
||||
'HX-Request': 'true'
|
||||
}
|
||||
});
|
||||
|
||||
if (resp.ok) {
|
||||
const redirect = resp.headers.get('HX-Redirect');
|
||||
if (redirect) window.location.href = redirect;
|
||||
else window.location.reload();
|
||||
} else {
|
||||
const text = await resp.text();
|
||||
alert('Import failed: ' + text);
|
||||
}
|
||||
} catch (err) {
|
||||
alert('Import error: ' + err);
|
||||
}
|
||||
}
|
||||
</script>
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add templates/watchlist.gohtml templates/watchlist_partial.gohtml
|
||||
git commit -m "feat: add export/import buttons and logic to watchlist UI"
|
||||
```
|
||||
@@ -1,20 +0,0 @@
|
||||
# Watchlist Import Improvement Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Wrap the watchlist import loop in a transaction and log skipped rows.
|
||||
|
||||
**Architecture:** Use `db.BeginTx` to start a transaction and ensure all operations within the loop use the transaction-aware `txQueries`.
|
||||
|
||||
**Tech Stack:** Go, standard library `database/sql` and `log`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Refactor ImportWatchlist to use Transaction
|
||||
|
||||
**Files:**
|
||||
- Modify: `/Users/mkelvers/dev/personal/mal/api/watchlist/service.go`
|
||||
|
||||
- [ ] **Step 1: Apply transaction wrapper and logging**
|
||||
|
||||
- [ ] **Step 2: Commit changes**
|
||||
@@ -1,41 +0,0 @@
|
||||
# Design Spec: AllAnime Client Decomposition
|
||||
|
||||
Decompose the monolithic `api/playback/allanime_client.go` into focused, maintainable units to improve readability, testability, and separation of concerns.
|
||||
|
||||
## 1. File Structure & Responsibilities
|
||||
|
||||
The current 1200-line file will be split into the following components:
|
||||
|
||||
### `pkg/net/utls/utls.go` (New)
|
||||
- **Responsibility:** Generic uTLS and HTTP2 networking utilities.
|
||||
- **Contents:** `utlsRoundTripper` struct and its `RoundTrip` implementation.
|
||||
- **Benefit:** Decouples complex networking logic from domain-specific client code.
|
||||
|
||||
### `api/playback/allanime_crypto.go` (New)
|
||||
- **Responsibility:** AES decryption and dynamic key discovery.
|
||||
- **Contents:** `decryptTobeparsed`, `getAESKey`, `fetchKeyFromForks`, and supporting validation/decryption logic.
|
||||
- **Benefit:** Isolates the "hacky" key-scraping and crypto logic from the clean API client.
|
||||
|
||||
### `api/playback/allanime_extractor.go` (New)
|
||||
- **Responsibility:** Parsing raw data and detecting media types.
|
||||
- **Contents:** `extractSourceURLsFromData`, `decodeSourceURL`, `detectStreamType`, `detectEmbedType`, and source-building helpers.
|
||||
- **Benefit:** Clean separation of data transformation logic.
|
||||
|
||||
### `api/playback/allanime_client.go` (Modified)
|
||||
- **Responsibility:** Main service interface and GraphQL communication.
|
||||
- **Contents:** `Search`, `GetEpisodes`, `GetEpisodeSources`, and `graphqlRequest`.
|
||||
- **Benefit:** The file becomes a thin wrapper around the API protocol.
|
||||
|
||||
## 2. Interface Stability
|
||||
- The existing `allAnimeClient` struct and its public methods (`Search`, `GetEpisodes`, etc.) will maintain their signatures.
|
||||
- Internal helper functions will be moved to the new files but remain within the `playback` package (except for the generic networking utility).
|
||||
|
||||
## 3. Verification Plan
|
||||
|
||||
### Automated Checks
|
||||
- `go test ./api/playback/...`: Ensure all existing tests pass after the split.
|
||||
- `go build ./...`: Verify no compilation errors due to visibility or package changes.
|
||||
- Add new unit tests in `api/playback/allanime_crypto_test.go` for the moved decryption logic.
|
||||
|
||||
### Manual Checks
|
||||
- Verify that anime playback and searching still function correctly in the application.
|
||||
@@ -1,38 +0,0 @@
|
||||
# Design Spec: Project Structure Cleanup and Standardization
|
||||
|
||||
Standardize the Go project structure, consolidate database packages, and remove redundant files to improve maintainability and idiomatic code patterns.
|
||||
|
||||
## 1. Database Package Consolidation
|
||||
|
||||
### Current State
|
||||
- `internal/db` contains `sqlc` generated code in package `database`.
|
||||
- `internal/db/sqlite/sqlite.go` is in a separate package for DB initialization.
|
||||
- Imports are scattered between `mal/internal/db` (often aliased) and `mal/internal/db/sqlite`.
|
||||
|
||||
### Changes
|
||||
- **Package Renaming:** Change package `database` to `db` in all `internal/db/*.go` files.
|
||||
- **sqlc configuration:** Update `sqlc.yaml` to use `package: db`.
|
||||
- **Flattening:** Move `internal/db/sqlite/sqlite.go` to `internal/db/sqlite.go`.
|
||||
- **Import Cleanup:** Update all consumers to use `mal/internal/db` directly.
|
||||
|
||||
## 2. Redundant File Removal
|
||||
|
||||
### Changes
|
||||
- Delete all `.go` files in `tmp/scripts/` (`fix_json.go`, `fix_proxy.go`, `test_jikan.go`, etc.).
|
||||
- Remove the empty directory `internal/db/sqlite` after moving the file.
|
||||
|
||||
## 3. Asset Build Optimization
|
||||
|
||||
### Changes
|
||||
- Investigate `dist/` duplication.
|
||||
- Update `package.json` build scripts if necessary to ensure `dist/` has a flat, clean structure matching what `NewRouter` expects.
|
||||
|
||||
## 4. Verification Plan
|
||||
|
||||
### Automated Checks
|
||||
- `go build ./...`: Verify all Go imports and package renames.
|
||||
- `bun run typecheck`: Verify TS/JS integrity.
|
||||
- `sqlc generate`: Verify that regeneration doesn't break the new package naming.
|
||||
|
||||
### Manual Checks
|
||||
- Verify server starts and database migrations apply correctly.
|
||||
@@ -1,68 +0,0 @@
|
||||
# Design Spec: Generic Rate Limiter Refactor
|
||||
|
||||
Refactor the current global-state rate limiter into a struct-based implementation that supports multiple instances with different configurations, improving testability and flexibility.
|
||||
|
||||
## 1. Core Architecture
|
||||
|
||||
### Current State
|
||||
- `pkg/middleware/ratelimit.go` uses global variables (`visitors`, `mu`, `quit`).
|
||||
- Cleanup logic is started automatically via `init()`.
|
||||
- Limits are hardcoded (5 attempts per minute).
|
||||
|
||||
### Changes
|
||||
- **Limiter Struct:** Encapsulate state within a `Limiter` struct.
|
||||
- **Explicit Configuration:** Pass limits and windows via a `Config` struct.
|
||||
- **Context-aware Cleanup:** Use `context.Context` to manage the cleanup goroutine lifecycle instead of a global `quit` channel.
|
||||
|
||||
## 2. Component Design
|
||||
|
||||
### `Limiter` Struct
|
||||
```go
|
||||
type visitor struct {
|
||||
attempts int
|
||||
lastSeen time.Time
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
MaxAttempts int
|
||||
Window time.Duration
|
||||
}
|
||||
|
||||
type Limiter struct {
|
||||
mu sync.Mutex
|
||||
visitors map[string]*visitor
|
||||
config Config
|
||||
}
|
||||
```
|
||||
|
||||
### Initialization
|
||||
```go
|
||||
func NewLimiter(cfg Config) *Limiter {
|
||||
return &Limiter{
|
||||
visitors: make(map[string]*visitor),
|
||||
config: cfg,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Middleware Methods
|
||||
- `Middleware(next http.Handler) http.Handler`: Generic rate limiting.
|
||||
- `AuthMiddleware(next http.Handler) http.Handler`: Specific logic for auth (redirects with error params).
|
||||
|
||||
## 3. Integration Plan
|
||||
|
||||
- Instantiate the `Limiter` in `cmd/server/main.go`.
|
||||
- Pass the `Limiter` or its middleware to `internal/server/NewRouter`.
|
||||
- Update `internal/server/routes.go` to use the new instance-based middleware.
|
||||
|
||||
## 4. Verification Plan
|
||||
|
||||
### Automated Checks
|
||||
- **Unit Tests:** Create `pkg/middleware/ratelimit_test.go` to verify:
|
||||
- Multiple IPs are tracked independently.
|
||||
- Limits are enforced correctly.
|
||||
- Attempts reset after the window expires.
|
||||
- `go test ./pkg/middleware/...`: Run the new tests.
|
||||
|
||||
### Manual Checks
|
||||
- Verify that exceeding login attempts still triggers the `rate_limited` redirect.
|
||||
@@ -1,50 +0,0 @@
|
||||
# Spec: Watchlist CSV Export and Import
|
||||
|
||||
## Overview
|
||||
Add functionality to export the user's watchlist to a CSV file and import it back. This allows for easy backup, external editing, and migration of watchlist data.
|
||||
|
||||
## Requirements
|
||||
- **Export**: Generate a CSV file containing `anime_id`, `status`, `current_episode`, and `current_time_seconds`.
|
||||
- **Import**: Upload a CSV file to update or add watchlist entries.
|
||||
- **Data Integrity**:
|
||||
- Do not export `created_at` or `updated_at`.
|
||||
- On import, overwrite existing entries with the same `anime_id`.
|
||||
- Ensure anime exists in the local database before adding to watchlist (fetch from Jikan if necessary).
|
||||
- **UI**: Add Export and Import buttons to the watchlist page header.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Frontend
|
||||
- **Template**: Update `templates/watchlist.gohtml` (and partial) to include Export/Import buttons.
|
||||
- **Export Implementation**:
|
||||
- Use JavaScript to iterate over the `AllEntries` data passed to the template.
|
||||
- Create a CSV string.
|
||||
- Trigger a browser download using a `Blob` and a temporary link element.
|
||||
- **Import Implementation**:
|
||||
- Use a hidden `<input type="file" accept=".csv">`.
|
||||
- When a file is selected, use `FormData` to upload it to the server via `fetch`.
|
||||
- Show a simple alert or refresh the page on success.
|
||||
|
||||
### Backend
|
||||
- **Endpoint**: `POST /api/watchlist/import`
|
||||
- **Service**: Add `ImportWatchlist(ctx, userID, reader)` to `api/watchlist/service.go`.
|
||||
- **Logic**:
|
||||
- Parse CSV using `encoding/csv`.
|
||||
- For each record:
|
||||
- Validate data types.
|
||||
- Call `ensureAnimeExists` (existing service method).
|
||||
- Use `UpsertWatchListEntry` to save/update.
|
||||
- Process the entire file in a transaction if possible, or gracefully skip malformed rows.
|
||||
|
||||
## Data Format (CSV)
|
||||
Headers: `anime_id,status,current_episode,current_time_seconds`
|
||||
Example:
|
||||
```csv
|
||||
123,watching,5,450.5
|
||||
456,completed,12,0
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
1. Clicking "Export" downloads a valid CSV file.
|
||||
2. Modifying the CSV and clicking "Import" correctly updates the watchlist in the UI and database.
|
||||
3. New anime added via CSV are correctly fetched from Jikan and added to the database.
|
||||
Reference in New Issue
Block a user