melosh/mal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'upstream/main' into main
All checks were successful
Build and Push Container Image / build-and-push (push) Successful in 9m21s
Details

Browse Source
This commit is contained in:
Milas Holsting
2026-06-15 21:37:41 +02:00
parent 12076f4cbb 6841f5c55a
commit 8fd7c1104c
150 changed files with 7982 additions and 5572 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
.golangci.yml
View File

@@ -3,15 +3,58 @@ version: "2"
linters:
default: none
enable:
- bodyclose
- copyloopvar
- cyclop
- dogsled
- dupl
- errcheck
- funlen
- gocognit
- gocritic
- gocyclo
- govet
- ineffassign
- maintidx
- makezero
- nakedret
- nilerr
- noctx
- prealloc
- predeclared
- revive
- staticcheck
- unconvert
- unparam
- unused
- usestdlibvars
- wastedassign
- whitespace
settings:
gocritic:
disable-all: true
enabled-checks:
- appendCombine
- boolExprSimplify
- commentedOutCode
- commentedOutImport
- deferUnlambda
- dupBranchBody
- dupImport
- dupSubExpr
- emptyDecl
- emptyFallthrough
- emptyStringTest
- equalFold
- redundantSprint
- regexpPattern
- stringConcatSimplify
- typeUnparen
- underef
- unlambda
- unnecessaryBlock
- unnecessaryDefer
- unslice
revive:
enable-all-rules: false
rules:
@@ -39,7 +82,7 @@ linters:
- third_party$
- builtin$
- examples$
- node_modules$
- node_modules/
issues:
max-issues-per-linter: 0

6
Dockerfile
View File

@@ -18,9 +18,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN curl -fsSL https://bun.sh/install | bash
ENV PATH="/root/.bun/bin:${PATH}"
# Install sqlc for code generation
RUN go install github.com/sqlc-dev/sqlc/cmd/sqlc@v1.30.0
ENV GOPROXY=direct
COPY go.mod go.sum ./
RUN go mod download
@@ -34,9 +31,6 @@ COPY . .
# Ensure dist is clean at build time (belt + suspenders)
RUN rm -rf dist/ && bun run build:assets
# Generate sqlc code
RUN sqlc generate
# Build the server and CLI tools
RUN go build -ldflags="-s -w" -o main_server ./cmd/server
RUN go build -ldflags="-s -w" -o create-user ./cmd/user

70
README.md
View File

@@ -1,10 +1,7 @@
# MyAnimeList
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="/static/assets/readme-logo-dark.svg" />
<img src="/static/assets/readme-logo-light.svg" alt="MyAnimeList logo" width="120" />
</picture>
<img src="/static/assets/logo.png" alt="MyAnimeList logo" width="120" />
</p>
<p align="center">
@@ -12,60 +9,47 @@
<img alt="SQLite" src="https://img.shields.io/badge/database-sqlite-003B57?style=flat-square&logo=sqlite" />
<img alt="Tailwind" src="https://img.shields.io/badge/tailwind-4-06D6D4?style=flat-square&logo=tailwindcss" />
<img alt="HTMX" src="https://img.shields.io/badge/htmx-partial--updates-3366CC?style=flat-square" />
<img alt="License" src="https://img.shields.io/badge/license-MIT-green?style=flat-square" />
</p>
---
MyAnimeList is a small self-hosted anime tracker and playback app. It keeps the catalog, watchlist, progress tracking, and player in one place, backed by a single SQLite database and a single Go server.
I built this because nothing else felt right. Every tracker I tried had decent pieces but the whole never clicked — awkward UI, missing features, or it just got in the way of actually watching anime. So I built one that fits how I work.
Most of the UI is rendered on the server. HTMX handles lightweight updates like search, pagination, and watchlist changes, while TypeScript is kept for the parts that need real browser state: the video player, command palette, theme handling, and skip segment editor. The app also includes local users, API tokens, subtitle support, playlist rewriting, provider integrations, migrations, and startup data fixes.
It is a self-hosted Go server that streams anime through a proxy layer, catalogs metadata, and tracks your progress.
## Running
The frontend is Tailwind CSS v4 with HTMX handling pagination, infinite scroll, search, and watchlist interactions. TypeScript only steps in where HTMX cannot — the video player, command palette bound to Cmd+K, skip segment editor, theme toggling with system preference detection, and custom UI components. Everything lives in one process, one SQLite database, one deployment.
## Repository structure
| Path | Purpose |
| ----------------- | ------------------------------------------------ |
| `api/*` | Feature routes: anime, auth, playback, watchlist |
| `cmd/server` | Application entrypoint and CLI commands |
| `cmd/user` | User management CLI (create, update, delete) |
| `integrations/*` | External API clients and scraping |
| `internal/*` | Core services: db, middleware, server, worker |
| `pkg/middleware` | Generic HTTP middleware |
| `templates/*` | Server-rendered HTML templates |
| `migrations` | Schema evolution (20 migrations) |
| `static` / `dist` | Frontend assets |
## Running locally
Requires Go `1.25+`, Bun, and [just](https://github.com/casey/just). Migrations run on startup. Configuration lives in environment variables — see `cmd/server/main.go` for the full list.
An optional API key from [animeschedule.net](https://animeschedule.net) can be used for the schedule board to enable English titles and improve performance. Create an account, generate a token under your profile, and set it as `ANIMESCHEDULE_API_TOKEN`.
Requires Go `1.25+`, Bun, [`just`](https://github.com/casey/just), and a C compiler for SQLite.
```bash
bun install
just build
go run ./cmd/user <username> <password>
just dev
```
## Quality checks
The app starts on `http://localhost:3000` by default. Configuration comes from environment variables, and a local `.env` file is loaded automatically. The most useful options are `PORT`, `DATABASE_FILE`, `PLAYBACK_PROXY_SECRET`, `EPISODE_AVAILABILITY_MODE`, and `ANIMESCHEDULE_API_TOKEN`.
## Development
The codebase is split between Go feature packages, external integrations, server-rendered templates, and a small frontend asset pipeline. `cmd/server` starts the web app, `cmd/user` contains local admin tools, `internal` holds the application modules, `integrations` holds provider clients, and `templates`, `static`, and `dist` contain the UI.
The common development commands are in the `justfile`.
```bash
gofmt -l .
go test ./...
go build -o server ./cmd/server
golangci-lint run ./...
go mod tidy
go test -race ./...
bunx oxfmt --check
bun run lint:ts
bun run typecheck
bun run build:assets
docker build -t mal:ci .
just fmt
just test
just lint-go
just lint-ts
just typecheck
just build
```
## Contributing
Run the full local check with:
Bug reports and pull requests are welcome. This is a personal project, so there is no strict roadmap or issue triage cycle. If something is broken or missing, open an issue or send a PR.
```bash
just check
```
## License
MIT. See `LICENSE`.
MIT. See [`LICENSE`](LICENSE).

3
bun.lock
View File

@@ -5,6 +5,7 @@
"": {
"name": "myanimelist-ui",
"dependencies": {
"hls.js": "^1.6.16",
"htmx.org": "1.9.12",
},
"devDependencies": {
@@ -185,6 +186,8 @@
"graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="],
"hls.js": ["hls.js@1.6.16", "", {}, "sha512-VSIRpLfRwlAAdGL4wiTucx2ScRipo0ed1FBatWkyt832jC4CReKstga6yIhYVwGu9LOBjuX9wzmRMeQdBJtzEA=="],
"htmx.org": ["htmx.org@1.9.12", "", {}, "sha512-VZAohXyF7xPGS52IM8d1T1283y+X4D+Owf3qY1NZ9RuBypyu9l8cGsxUMAG5fEAb/DhT7rDoJ9Hpu5/HxFD3cw=="],
"is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="],

40
cmd/user/main.go
View File

@@ -3,6 +3,7 @@ package main
import (
"bufio"
"context"
"database/sql"
"errors"
"fmt"
@@ -36,6 +37,8 @@ func main() {
}
func run(dbConn *sql.DB, args []string) int {
ctx := context.Background()
cmd, err := parseArgs(args)
if err != nil {
observability.Warn("cli_usage", "cmd/user", "invalid arguments", map[string]any{"argc": len(args)}, err)
@@ -45,13 +48,13 @@ func run(dbConn *sql.DB, args []string) int {
switch cmd.kind {
case commandUpdateAvatar:
updateAvatars(dbConn)
updateAvatars(ctx, dbConn)
return 0
case commandRunFixes:
runFixes(dbConn)
runFixes(ctx, dbConn)
return 0
case commandCreateOrUpdateUser:
if err := createOrUpdateUser(dbConn, cmd.username, cmd.password); err != nil {
if err := createOrUpdateUser(ctx, dbConn, cmd.username, cmd.password); err != nil {
return 1
}
return 0
@@ -100,8 +103,8 @@ func usage() string {
return "Usage: go run cmd/user/main.go <username> <password>\n go run cmd/user/main.go update-avatar\n go run cmd/user/main.go run-fixes"
}
func createOrUpdateUser(dbConn *sql.DB, username string, password string) error {
existingID, err := lookupUserID(dbConn, username)
func createOrUpdateUser(ctx context.Context, dbConn *sql.DB, username string, password string) error {
existingID, err := lookupUserID(ctx, dbConn, username)
if err != nil {
observability.Error("cli_user_lookup_failed", "cmd/user", "", map[string]any{"username": username}, err)
return err
@@ -112,23 +115,23 @@ func createOrUpdateUser(dbConn *sql.DB, username string, password string) error
fmt.Println("Operation cancelled.")
return nil
}
if err := updateUserPassword(dbConn, existingID, username, password); err != nil {
if err := updateUserPassword(ctx, dbConn, existingID, username, password); err != nil {
return err
}
fmt.Printf("Password for '%s' updated successfully!\n", username)
return nil
}
if err := createUser(dbConn, username, password); err != nil {
if err := createUser(ctx, dbConn, username, password); err != nil {
return err
}
fmt.Printf("User '%s' was created successfully!\n", username)
return nil
}
func lookupUserID(dbConn *sql.DB, username string) (string, error) {
func lookupUserID(ctx context.Context, dbConn *sql.DB, username string) (string, error) {
var id string
err := dbConn.QueryRow("SELECT id FROM user WHERE username = ?", username).Scan(&id)
err := dbConn.QueryRowContext(ctx, "SELECT id FROM user WHERE username = ?", username).Scan(&id)
if err == nil {
return id, nil
}
@@ -146,14 +149,14 @@ func promptConfirmOverwrite(username string) bool {
return response == "y" || response == "yes"
}
func updateUserPassword(dbConn *sql.DB, userID string, username string, password string) error {
func updateUserPassword(ctx context.Context, dbConn *sql.DB, userID string, username string, password string) error {
hash, err := bcrypt.GenerateFromPassword([]byte(password), 12)
if err != nil {
observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err)
return err
}
_, err = dbConn.Exec("UPDATE user SET password_hash = ? WHERE id = ?", string(hash), userID)
_, err = dbConn.ExecContext(ctx, "UPDATE user SET password_hash = ? WHERE id = ?", string(hash), userID)
if err != nil {
observability.Error("cli_user_password_update_failed", "cmd/user", "", map[string]any{"username": username}, err)
return err
@@ -161,7 +164,7 @@ func updateUserPassword(dbConn *sql.DB, userID string, username string, password
return nil
}
func createUser(dbConn *sql.DB, username string, password string) error {
func createUser(ctx context.Context, dbConn *sql.DB, username string, password string) error {
hash, err := bcrypt.GenerateFromPassword([]byte(password), 12)
if err != nil {
observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err)
@@ -170,7 +173,8 @@ func createUser(dbConn *sql.DB, username string, password string) error {
id := uuid.New().String()
avatarURL := internal.DefaultAvatarURL(username)
_, err = dbConn.Exec(
_, err = dbConn.ExecContext(
ctx,
"INSERT INTO user (id, username, password_hash, avatar_url) VALUES (?, ?, ?, ?)",
id,
username,
@@ -184,8 +188,8 @@ func createUser(dbConn *sql.DB, username string, password string) error {
return nil
}
func updateAvatars(dbConn *sql.DB) {
rows, err := dbConn.Query("SELECT id, username FROM user")
func updateAvatars(ctx context.Context, dbConn *sql.DB) {
rows, err := dbConn.QueryContext(ctx, "SELECT id, username FROM user")
if err != nil {
observability.Error("cli_users_list_failed", "cmd/user", "", nil, err)
os.Exit(1)
@@ -201,7 +205,7 @@ func updateAvatars(dbConn *sql.DB) {
}
avatarURL := internal.DefaultAvatarURL(username)
_, err := dbConn.Exec("UPDATE user SET avatar_url = ? WHERE id = ?", avatarURL, id)
_, err := dbConn.ExecContext(ctx, "UPDATE user SET avatar_url = ? WHERE id = ?", avatarURL, id)
if err != nil {
observability.Error("cli_user_avatar_update_failed", "cmd/user", "", map[string]any{"username": username}, err)
os.Exit(1)
@@ -217,13 +221,13 @@ func updateAvatars(dbConn *sql.DB) {
fmt.Printf("Updated avatars for %d user(s)\n", count)
}
func runFixes(dbConn *sql.DB) {
func runFixes(ctx context.Context, dbConn *sql.DB) {
if err := database.RunMigrationsAndFixes(dbConn); err != nil {
observability.Error("cli_run_migrations_and_fixes_failed", "cmd/user", "", nil, err)
os.Exit(1)
}
rows, err := dbConn.Query("SELECT id, applied_at FROM data_fixes ORDER BY id ASC")
rows, err := dbConn.QueryContext(ctx, "SELECT id, applied_at FROM data_fixes ORDER BY id ASC")
if err != nil {
observability.Error("cli_data_fixes_list_failed", "cmd/user", "", nil, err)
os.Exit(1)

2
integrations/animeschedule/animeschedule.go
View File

@@ -358,7 +358,7 @@ func fetchDocument(ctx context.Context, httpClient *http.Client, url string) (*g
return nil, url, err
}
return document, response.Request.URL.String(), nil
return document, response, nil
}
type timetableAnimeAPI struct {

178
integrations/jikan/client.go
View File

@@ -149,25 +149,43 @@ func jikanTraceEnabled() bool {
return traceEnabled
}
func logJikanCache(cacheKey string, source string, startedAt time.Time, err error) {
duration := time.Since(startedAt)
if !jikanTraceEnabled() && err == nil && source == "fresh" && duration < 50*time.Millisecond {
return
}
if !jikanTraceEnabled() && err == nil && source == "refresh" && duration < jikanSlowLogThreshold {
return
func shouldSkipJikanCacheLog(source string, duration time.Duration, err error) bool {
if jikanTraceEnabled() || err != nil {
return false
}
level := observability.LogLevelInfo
if source == "fresh" {
return duration < 50*time.Millisecond
}
if source == "refresh" {
return duration < jikanSlowLogThreshold
}
return false
}
func jikanCacheLogLevel(source string, err error) observability.LogLevel {
if err != nil {
level = observability.LogLevelError
} else if source != "fresh" && source != "refresh" {
return observability.LogLevelError
}
if source != "fresh" && source != "refresh" {
// Stale reads are expected sometimes, but worth tracking in logs.
level = observability.LogLevelWarn
return observability.LogLevelWarn
}
return observability.LogLevelInfo
}
func logJikanCache(cacheKey string, source string, startedAt time.Time, err error) {
duration := time.Since(startedAt)
if shouldSkipJikanCacheLog(source, duration, err) {
return
}
observability.LogJSON(
level,
jikanCacheLogLevel(source, err),
"jikan_cache",
"jikan",
"",
@@ -475,80 +493,114 @@ func (c *Client) fetchWithRetry(ctx context.Context, urlStr string, out any) err
for attempt := range maxRetries {
attempts = attempt + 1
select {
case <-ctx.Done():
return logAndReturn(0, fmt.Errorf("request canceled while retrying jikan request: %w", ctx.Err()))
default:
}
if err := c.waitRateLimit(ctx); err != nil {
if err := c.prepareRetryAttempt(ctx); err != nil {
return logAndReturn(0, err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, urlStr, nil)
resp, err := c.doRequest(ctx, urlStr)
if err != nil {
return logAndReturn(0, fmt.Errorf("failed to create jikan request: %w", err))
}
req.Header.Set("User-Agent", netutil.Generic)
resp, err := c.httpClient.Do(req)
if err != nil {
if errors.Is(err, context.Canceled) {
return logAndReturn(0, fmt.Errorf("request canceled while retrying jikan request: %w", err))
}
if attempt < maxRetries-1 && IsRetryableError(err) {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return logAndReturn(0, retryErr)
}
retry, requestErr := handleRequestRetry(ctx, err, attempt, maxRetries)
if retry {
continue
}
return logAndReturn(0, fmt.Errorf("jikan api error: %w", err))
return logAndReturn(0, requestErr)
}
statusCode, retry, err := handleResponseRetry(ctx, resp, urlStr, out, attempt, maxRetries)
if retry {
continue
}
return logAndReturn(statusCode, err)
}
return logAndReturn(0, fmt.Errorf("max retries exceeded for %s", urlStr))
}
func (c *Client) prepareRetryAttempt(ctx context.Context) error {
select {
case <-ctx.Done():
return fmt.Errorf("request canceled while retrying jikan request: %w", ctx.Err())
default:
}
return c.waitRateLimit(ctx)
}
func (c *Client) doRequest(ctx context.Context, urlStr string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, urlStr, nil)
if err != nil {
return nil, fmt.Errorf("failed to create jikan request: %w", err)
}
req.Header.Set("User-Agent", netutil.Generic)
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
return resp, nil
}
func handleRequestRetry(ctx context.Context, err error, attempt int, maxRetries int) (bool, error) {
if errors.Is(err, context.Canceled) {
return false, fmt.Errorf("request canceled while retrying jikan request: %w", err)
}
if attempt >= maxRetries-1 || !IsRetryableError(err) {
return false, fmt.Errorf("jikan api error: %w", err)
}
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return false, retryErr
}
return true, nil
}
func handleResponseRetry(ctx context.Context, resp *http.Response, urlStr string, out any, attempt int, maxRetries int) (int, bool, error) {
if resp.StatusCode != http.StatusOK {
apiErr := &APIError{StatusCode: resp.StatusCode, URL: urlStr}
retryable := isRetryableStatus(resp.StatusCode)
return handleStatusRetry(ctx, resp, urlStr, out, attempt, maxRetries)
}
err := json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
if err == nil {
return resp.StatusCode, false, nil
}
if attempt < maxRetries-1 {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return resp.StatusCode, false, retryErr
}
return resp.StatusCode, true, nil
}
return resp.StatusCode, false, fmt.Errorf("failed to decode jikan response: %w", err)
}
func handleStatusRetry(ctx context.Context, resp *http.Response, urlStr string, out any, attempt int, maxRetries int) (int, bool, error) {
statusCode := resp.StatusCode
apiErr := &APIError{StatusCode: statusCode, URL: urlStr}
retryAfter := time.Duration(0)
if parsed, ok := parseRetryAfter(resp.Header.Get("Retry-After")); ok {
retryAfter = parsed
}
if retryable && attempt < maxRetries-1 {
if isRetryableStatus(statusCode) && attempt < maxRetries-1 {
_ = resp.Body.Close()
delay := max(retryAfter, retryDelay(attempt))
if retryErr := waitForRetry(ctx, delay); retryErr != nil {
return logAndReturn(resp.StatusCode, retryErr)
if retryErr := waitForRetry(ctx, max(retryAfter, retryDelay(attempt))); retryErr != nil {
return statusCode, false, retryErr
}
continue
return statusCode, true, nil
}
// Best-effort decode (often useful for debugging), but still treat non-200 as error.
_ = json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
return logAndReturn(resp.StatusCode, apiErr)
}
err = json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
if err == nil {
return logAndReturn(resp.StatusCode, nil)
}
if attempt < maxRetries-1 {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return logAndReturn(resp.StatusCode, retryErr)
}
continue
}
return logAndReturn(resp.StatusCode, fmt.Errorf("failed to decode jikan response: %w", err))
}
return logAndReturn(0, fmt.Errorf("max retries exceeded for %s", urlStr))
return statusCode, false, apiErr
}
func metricsEndpoint(urlStr string) string {

89
integrations/jikan/client_test.go
View File

@@ -23,42 +23,13 @@ func (fn roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
}
func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
sqlDB, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
sqlDB := newTestCacheDB(t)
defer sqlDB.Close()
sqlDB.SetMaxOpenConns(1)
_, err = sqlDB.Exec(`
CREATE TABLE jikan_cache (
key TEXT PRIMARY KEY,
data TEXT NOT NULL,
expires_at DATETIME NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
`)
if err != nil {
t.Fatalf("create cache table: %v", err)
}
queries := db.New(sqlDB)
client := NewClient(config.Config{}, queries, observability.NewMetrics())
stale := TopAnimeResponse{Data: []Anime{{MalID: 1, Title: "stale"}}}
staleBytes, err := json.Marshal(stale)
if err != nil {
t.Fatalf("marshal stale response: %v", err)
}
_, err = sqlDB.Exec(
`INSERT INTO jikan_cache (key, data, expires_at) VALUES (?, ?, ?)`,
"top:1",
string(staleBytes),
time.Now().Add(-time.Hour),
)
if err != nil {
t.Fatalf("insert stale cache: %v", err)
}
insertCachedResponse(t, sqlDB, "top:1", stale, time.Now().Add(-time.Hour))
client.httpClient = &http.Client{
Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
@@ -78,11 +49,63 @@ func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
if len(got.Data) != 1 || got.Data[0].Title != "stale" {
t.Fatalf("got %+v, want stale cache response", got.Data)
}
waitForFreshCache(t, sqlDB, client, "top:1")
}
func newTestCacheDB(t *testing.T) *sql.DB {
t.Helper()
ctx := context.Background()
sqlDB, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
sqlDB.SetMaxOpenConns(1)
_, err = sqlDB.ExecContext(ctx, `
CREATE TABLE jikan_cache (
key TEXT PRIMARY KEY,
data TEXT NOT NULL,
expires_at DATETIME NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
`)
if err != nil {
sqlDB.Close()
t.Fatalf("create cache table: %v", err)
}
return sqlDB
}
func insertCachedResponse(t *testing.T, sqlDB *sql.DB, key string, value TopAnimeResponse, expiresAt time.Time) {
t.Helper()
ctx := context.Background()
encoded, err := json.Marshal(value)
if err != nil {
t.Fatalf("marshal cached response: %v", err)
}
_, err = sqlDB.ExecContext(
ctx,
`INSERT INTO jikan_cache (key, data, expires_at) VALUES (?, ?, ?)`,
key,
string(encoded),
expiresAt,
)
if err != nil {
t.Fatalf("insert cached response: %v", err)
}
}
func waitForFreshCache(t *testing.T, sqlDB *sql.DB, client *Client, key string) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
var refreshed TopAnimeResponse
if client.getCache(context.Background(), "top:1", &refreshed) && len(refreshed.Data) == 1 && refreshed.Data[0].Title == "fresh" {
if client.getCache(context.Background(), key, &refreshed) && len(refreshed.Data) == 1 && refreshed.Data[0].Title == "fresh" {
return
}
time.Sleep(10 * time.Millisecond)
@@ -90,6 +113,6 @@ func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
var rawData string
var rawExpires string
_ = sqlDB.QueryRow(`SELECT data, expires_at FROM jikan_cache WHERE key = ?`, "top:1").Scan(&rawData, &rawExpires)
_ = sqlDB.QueryRowContext(context.Background(), `SELECT data, expires_at FROM jikan_cache WHERE key = ?`, key).Scan(&rawData, &rawExpires)
t.Fatalf("cache was not refreshed asynchronously; data=%s expires_at=%s", rawData, rawExpires)
}

6
integrations/jikan/episodes.go
View File

@@ -3,6 +3,8 @@ package jikan
import (
"context"
"fmt"
"net/url"
"strconv"
"sync"
"time"
)
@@ -15,7 +17,9 @@ func (c *Client) GetEpisodes(ctx context.Context, animeID int, page int) (Episod
cacheKey := fmt.Sprintf("anime:%d:episodes:%d", animeID, page)
var result EpisodesResponse
reqURL := fmt.Sprintf("%s/anime/%d/episodes?page=%d", c.baseURL, animeID, page)
params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, fmt.Sprintf("/anime/%d/episodes", animeID), params)
err := c.getWithCache(ctx, cacheKey, 12*time.Hour, reqURL, &result)
return result, err

6
integrations/jikan/more.go
View File

@@ -3,6 +3,8 @@ package jikan
import (
"context"
"fmt"
"net/url"
"strconv"
)
func (c *Client) GetAnimeStaff(ctx context.Context, id int) ([]StaffEntry, error) {
@@ -46,7 +48,9 @@ func (c *Client) GetAnimeReviews(ctx context.Context, id int, page int) ([]Revie
page = 1
}
url := fmt.Sprintf("%s/anime/%d/reviews?page=%d", c.baseURL, id, page)
params := url.Values{}
params.Set("page", strconv.Itoa(page))
url := buildRequestURL(c.baseURL, fmt.Sprintf("/anime/%d/reviews", id), params)
cacheKey := fmt.Sprintf("anime:reviews:%d:%d", id, page)
var resp ReviewsResponse

9
integrations/jikan/producers.go
View File

@@ -56,10 +56,11 @@ func (c *Client) GetProducers(ctx context.Context, query string, page int, limit
func (c *Client) fetchProducersPage(ctx context.Context, query string, page int, limit int) (ProducerListResult, error) {
q := strings.TrimSpace(query)
cacheKey := fmt.Sprintf("producers:%s:%d:%d", q, page, limit)
reqURL := fmt.Sprintf("%s/producers?page=%d&limit=%d", c.baseURL, page, limit)
if q != "" {
reqURL += "&q=" + url.QueryEscape(q)
}
params := url.Values{}
params.Set("page", strconv.Itoa(page))
params.Set("limit", strconv.Itoa(limit))
setQueryValue(params, "q", q)
reqURL := buildRequestURL(c.baseURL, "/producers", params)
var result ProducersResponse
if err := c.getWithCache(ctx, cacheKey, producerCacheTTL, reqURL, &result); err != nil {

43
integrations/jikan/query_params.go Normal file
View File

@@ -0,0 +1,43 @@
package jikan
import (
"fmt"
"net/url"
"strconv"
)
func buildRequestURL(baseURL, path string, params url.Values) string {
encoded := params.Encode()
if encoded == "" {
return fmt.Sprintf("%s%s", baseURL, path)
}
return fmt.Sprintf("%s%s?%s", baseURL, path, encoded)
}
func setQueryValue(values url.Values, key, value string) {
if value == "" {
values.Del(key)
return
}
values.Set(key, value)
}
func setPositiveIntQueryValue(values url.Values, key string, value int) {
if value <= 0 {
values.Del(key)
return
}
values.Set(key, strconv.Itoa(value))
}
func setTrueQueryValue(values url.Values, key string, enabled bool) {
if !enabled {
values.Del(key)
return
}
values.Set(key, "true")
}

120
integrations/jikan/relations.go
View File

@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"net/http"
"sort"
"strings"
"time"
@@ -20,6 +21,22 @@ const chiakiWatchOrderURL = "https://chiaki.site/?/tools/watch_order/id/%d"
const watchOrderCacheTTL = time.Hour * 24
const maxWatchOrderEntries = 120 // cap to prevent huge relation chains
type WatchOrderMode string
const (
WatchOrderModeMain WatchOrderMode = "main"
WatchOrderModeComplete WatchOrderMode = "complete"
)
func NormalizeWatchOrderMode(value string) WatchOrderMode {
switch WatchOrderMode(strings.ToLower(strings.TrimSpace(value))) {
case WatchOrderModeComplete:
return WatchOrderModeComplete
default:
return WatchOrderModeMain
}
}
// watchOrderTypeLabel normalizes watch order type to display-friendly labels.
func watchOrderTypeLabel(value string) string {
normalized := strings.ToLower(strings.TrimSpace(value))
@@ -28,17 +45,35 @@ func watchOrderTypeLabel(value string) string {
return "TV"
case "movie":
return "Movie"
case "ona":
return "ONA"
case "ova":
return "OVA"
default:
return strings.TrimSpace(value)
}
}
// isAllowedWatchOrderType returns true only for TV and Movie types (filters out specials, etc).
func isTVWatchOrderType(value string) bool {
return strings.EqualFold(strings.TrimSpace(value), "tv")
}
// isAllowedWatchOrderType returns true for the default uncluttered watch order types.
func isAllowedWatchOrderType(value string) bool {
normalized := strings.ToLower(strings.TrimSpace(value))
return normalized == "tv" || normalized == "movie"
}
func hasTVWatchOrderEntry(entries []watchorder.WatchOrderEntry) bool {
for _, entry := range entries {
if isTVWatchOrderType(entry.Type) {
return true
}
}
return false
}
func relationCacheKey(id int) string {
return fmt.Sprintf("relations:watch-order:%d", id)
}
@@ -52,7 +87,7 @@ func (c *Client) refreshWatchOrder(ctx context.Context, id int) (watchorder.Watc
result, err := watchorder.FetchWatchOrder(requestCtx, c.httpClient, watchOrderURL)
if err != nil {
var statusError *watchorder.HTTPStatusError
if errors.As(err, &statusError) && statusError.StatusCode == 404 {
if errors.As(err, &statusError) && statusError.StatusCode == http.StatusNotFound {
return watchorder.WatchOrderResult{}, watchorder.ErrWatchOrderNotFound
}
if errors.Is(err, watchorder.ErrWatchOrderMarkupNotFound) {
@@ -148,13 +183,11 @@ func (c *Client) currentOnlyRelation(ctx context.Context, id int) ([]RelationEnt
}}, nil
}
// GetFullRelations returns related anime based on watch order, with parallel fetching (3 concurrent).
func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry, error) {
result, err := c.getWatchOrder(ctx, id)
if err != nil {
func (c *Client) handleWatchOrderError(ctx context.Context, id int, err error) ([]RelationEntry, error) {
if errors.Is(err, watchorder.ErrWatchOrderNotFound) {
return c.currentOnlyRelation(ctx, id)
}
observability.Warn(
"relations_watch_order_fallback_current_only",
"jikan",
@@ -164,34 +197,40 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
},
err,
)
return c.currentOnlyRelation(ctx, id)
}
type fetchResult struct {
index int
anime Anime
entry watchorder.WatchOrderEntry
}
var allowedEntries []watchorder.WatchOrderEntry
func buildAllowedWatchOrderEntries(result watchorder.WatchOrderResult, mode WatchOrderMode) ([]watchorder.WatchOrderEntry, map[int]bool) {
allowedEntries := make([]watchorder.WatchOrderEntry, 0, len(result.WatchOrder))
seen := make(map[int]bool)
shouldIncludeAllTypes := mode == WatchOrderModeComplete || !hasTVWatchOrderEntry(result.WatchOrder)
for _, entry := range result.WatchOrder {
if len(allowedEntries) >= maxWatchOrderEntries {
break
}
if !isAllowedWatchOrderType(entry.Type) || seen[entry.ID] {
if seen[entry.ID] {
continue
}
if !shouldIncludeAllTypes && !isAllowedWatchOrderType(entry.Type) {
continue
}
seen[entry.ID] = true
allowedEntries = append(allowedEntries, entry)
}
return allowedEntries, seen
}
func (c *Client) fetchRelationResults(ctx context.Context, entries []watchorder.WatchOrderEntry) []fetchResult {
g, gCtx := errgroup.WithContext(ctx)
g.SetLimit(3)
results := make(chan fetchResult, len(allowedEntries))
results := make(chan fetchResult, len(entries))
for i, entry := range allowedEntries {
for i, entry := range entries {
g.Go(func() error {
anime, err := c.GetAnimeByID(gCtx, entry.ID)
if err != nil {
@@ -201,10 +240,12 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
c.EnqueueAnimeFetchRetry(gCtx, entry.ID, err)
return nil
}
select {
case results <- fetchResult{index: i, anime: anime, entry: entry}:
case <-gCtx.Done():
}
return nil
})
}
@@ -214,18 +255,21 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
close(results)
}()
fetched := make([]fetchResult, 0, len(allowedEntries))
fetched := make([]fetchResult, 0, len(entries))
for res := range results {
fetched = append(fetched, res)
}
// Re-sort because they might have finished out of order
sort.Slice(fetched, func(i, j int) bool {
return fetched[i].index < fetched[j].index
})
relations := make([]RelationEntry, 0, len(fetched)+1)
for _, res := range fetched {
return fetched
}
func buildRelationsFromResults(results []fetchResult, id int) []RelationEntry {
relations := make([]RelationEntry, 0, len(results)+1)
for _, res := range results {
relations = append(relations, RelationEntry{
Anime: res.anime,
Relation: watchOrderTypeLabel(res.entry.Type),
@@ -234,18 +278,46 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
})
}
if !seen[id] {
return relations
}
func (c *Client) ensureCurrentRelation(ctx context.Context, id int, seen map[int]bool, relations []RelationEntry) ([]RelationEntry, error) {
if seen[id] {
return relations, nil
}
currentAnime, err := c.GetAnimeByID(ctx, id)
if err != nil {
return nil, err
}
relations = append([]RelationEntry{{
return append([]RelationEntry{{
Anime: currentAnime,
Relation: "Current",
IsCurrent: true,
IsExtra: false,
}}, relations...)
}}, relations...), nil
}
type fetchResult struct {
index int
anime Anime
entry watchorder.WatchOrderEntry
}
// GetFullRelations returns related anime based on watch order, with parallel fetching (3 concurrent).
func (c *Client) GetFullRelations(ctx context.Context, id int, mode WatchOrderMode) ([]RelationEntry, error) {
result, err := c.getWatchOrder(ctx, id)
if err != nil {
return c.handleWatchOrderError(ctx, id, err)
}
allowedEntries, seen := buildAllowedWatchOrderEntries(result, mode)
fetched := c.fetchRelationResults(ctx, allowedEntries)
relations := buildRelationsFromResults(fetched, id)
relations, err = c.ensureCurrentRelation(ctx, id, seen, relations)
if err != nil {
return nil, err
}
if len(relations) == 0 {
@@ -257,6 +329,6 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
func (c *Client) WarmFullRelations(id int) {
c.runAsyncRefresh(func(ctx context.Context) {
_, _ = c.GetFullRelations(ctx, id)
_, _ = c.GetFullRelations(ctx, id, WatchOrderModeMain)
})
}

139
integrations/jikan/relations_test.go
View File

@@ -1,6 +1,9 @@
package jikan
import "testing"
import (
"mal/integrations/watchorder"
"testing"
)
func runBoolCases(t *testing.T, tests []struct {
name string
@@ -36,6 +39,138 @@ func TestIsAllowedWatchOrderType(t *testing.T) {
runBoolCases(t, tests, isAllowedWatchOrderType)
}
func TestNormalizeWatchOrderMode(t *testing.T) {
tests := []struct {
name string
input string
want WatchOrderMode
}{
{name: "empty defaults main", input: "", want: WatchOrderModeMain},
{name: "main", input: "main", want: WatchOrderModeMain},
{name: "complete", input: "complete", want: WatchOrderModeComplete},
{name: "case and whitespace", input: " COMPLETE ", want: WatchOrderModeComplete},
{name: "unknown defaults main", input: "everything", want: WatchOrderModeMain},
}
for _, testCase := range tests {
t.Run(testCase.name, func(t *testing.T) {
got := NormalizeWatchOrderMode(testCase.input)
if got != testCase.want {
t.Fatalf("expected %q, got %q", testCase.want, got)
}
})
}
}
func TestHasTVWatchOrderEntry(t *testing.T) {
tests := []struct {
name string
entries []watchorder.WatchOrderEntry
want bool
}{
{
name: "contains tv",
entries: []watchorder.WatchOrderEntry{
{ID: 1, Type: "Movie"},
{ID: 2, Type: " TV "},
},
want: true,
},
{
name: "ona only",
entries: []watchorder.WatchOrderEntry{
{ID: 1, Type: "ONA"},
{ID: 2, Type: "Special"},
},
want: false,
},
}
for _, testCase := range tests {
t.Run(testCase.name, func(t *testing.T) {
got := hasTVWatchOrderEntry(testCase.entries)
if got != testCase.want {
t.Fatalf("expected %v, got %v", testCase.want, got)
}
})
}
}
func TestBuildAllowedWatchOrderEntriesKeepsDefaultTypesWhenTVExists(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "TV"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "Movie"},
{ID: 4, Type: "ONA"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeMain)
if len(entries) != 2 {
t.Fatalf("expected 2 entries, got %d", len(entries))
}
if entries[0].ID != 1 || entries[1].ID != 3 {
t.Fatalf("unexpected entries: %+v", entries)
}
if !seen[1] || !seen[3] || seen[2] || seen[4] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestBuildAllowedWatchOrderEntriesIncludesAllTypesWhenNoTVExists(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "ONA"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "Movie"},
{ID: 1, Type: "ONA"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeMain)
if len(entries) != 3 {
t.Fatalf("expected 3 entries, got %d", len(entries))
}
if entries[0].ID != 1 || entries[1].ID != 2 || entries[2].ID != 3 {
t.Fatalf("unexpected entries: %+v", entries)
}
if !seen[1] || !seen[2] || !seen[3] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestBuildAllowedWatchOrderEntriesIncludesAllTypesInCompleteMode(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "TV"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "ONA"},
{ID: 4, Type: "Movie"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeComplete)
if len(entries) != 4 {
t.Fatalf("expected 4 entries, got %d", len(entries))
}
for index, entry := range entries {
wantID := index + 1
if entry.ID != wantID {
t.Fatalf("expected entry %d to have id %d, got %+v", index, wantID, entry)
}
}
if !seen[1] || !seen[2] || !seen[3] || !seen[4] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestWatchOrderTypeLabel(t *testing.T) {
tests := []struct {
name string
@@ -44,6 +179,8 @@ func TestWatchOrderTypeLabel(t *testing.T) {
}{
{name: "tv", input: "tv", want: "TV"},
{name: "movie", input: "movie", want: "Movie"},
{name: "ona", input: "ona", want: "ONA"},
{name: "ova", input: "ova", want: "OVA"},
{name: "trimmed passthrough", input: " tv special ", want: "tv special"},
}

70
integrations/jikan/search.go
View File

@@ -8,8 +8,7 @@ import (
"strings"
)
// SearchAdvanced performs a filtered anime search with type, status, ordering, genre filters, and studio (producer) filters.
func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (SearchResult, error) {
func normalizeSearchPagination(page, limit int) (int, int) {
if page < 1 {
page = 1
}
@@ -17,46 +16,47 @@ func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, o
limit = 0
}
genresParam := ""
if len(genres) > 0 {
return page, limit
}
func joinGenreIDs(genres []int) string {
if len(genres) == 0 {
return ""
}
ids := make([]string, len(genres))
for i, g := range genres {
ids[i] = strconv.Itoa(g)
}
genresParam = strings.Join(ids, ",")
return strings.Join(ids, ",")
}
func buildAdvancedSearchURL(baseURL, query, animeType, status, orderBy, sort, genres string, studioID int, sfw bool, page, limit int) string {
params := url.Values{}
params.Set("page", strconv.Itoa(page))
setTrueQueryValue(params, "sfw", sfw)
setQueryValue(params, "q", query)
setQueryValue(params, "type", animeType)
setQueryValue(params, "status", status)
setPositiveIntQueryValue(params, "producers", studioID)
setQueryValue(params, "order_by", orderBy)
setQueryValue(params, "sort", sort)
setQueryValue(params, "genres", genres)
setPositiveIntQueryValue(params, "limit", limit)
return buildRequestURL(baseURL, "/anime", params)
}
// SearchAdvanced performs a filtered anime search with type, status, ordering, genre filters, and studio (producer) filters.
func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (SearchResult, error) {
page, limit = normalizeSearchPagination(page, limit)
genresParam := joinGenreIDs(genres)
cacheKey := fmt.Sprintf("search:%s:%s:%s:%s:%s:%s:%d:%v:%d:%d", query, animeType, status, orderBy, sort, genresParam, studioID, sfw, page, limit)
var result SearchResponse
reqURL := fmt.Sprintf("%s/anime?page=%d", c.baseURL, page)
if sfw {
reqURL += "&sfw=true"
}
if query != "" {
reqURL += "&q=" + url.QueryEscape(query)
}
if animeType != "" {
reqURL += "&type=" + url.QueryEscape(animeType)
}
if status != "" {
reqURL += "&status=" + url.QueryEscape(status)
}
if studioID > 0 {
reqURL += "&producers=" + strconv.Itoa(studioID)
}
if orderBy != "" {
reqURL += "&order_by=" + url.QueryEscape(orderBy)
}
if sort != "" {
reqURL += "&sort=" + url.QueryEscape(sort)
}
if genresParam != "" {
reqURL += "&genres=" + genresParam
}
if limit > 0 {
reqURL += fmt.Sprintf("&limit=%d", limit)
}
reqURL := buildAdvancedSearchURL(c.baseURL, query, animeType, status, orderBy, sort, genresParam, studioID, sfw, page, limit)
if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil {
return SearchResult{}, err
@@ -76,7 +76,9 @@ func (c *Client) GetTopAnime(ctx context.Context, page int) (TopAnimeResult, err
cacheKey := fmt.Sprintf("top:%d", page)
var result TopAnimeResponse
reqURL := fmt.Sprintf("%s/top/anime?page=%d", c.baseURL, page)
params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, "/top/anime", params)
if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil {
return TopAnimeResult{}, err

143
integrations/jikan/seasons.go
View File

@@ -5,6 +5,8 @@ import (
"encoding/json"
"fmt"
"math/rand"
"net/url"
"strconv"
"time"
)
@@ -30,7 +32,9 @@ func (c *Client) getSeasonList(ctx context.Context, page int, season string) (To
cacheKey := fmt.Sprintf("seasons_%s:%d", season, page)
var result TopAnimeResponse
reqURL := fmt.Sprintf("%s/seasons/%s?page=%d", c.baseURL, season, page)
params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, fmt.Sprintf("/seasons/%s", season), params)
err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result)
if err != nil {
@@ -44,78 +48,121 @@ func (c *Client) getSeasonList(ctx context.Context, page int, season string) (To
}
// seedRandomPool seeds the in-memory pool of random anime
func (c *Client) seedRandomPool(ctx context.Context) error {
func (c *Client) seedRandomPool(ctx context.Context) {
if !c.markRandomPoolInitialized() {
return
}
c.loadCachedRandomPool(ctx)
// Fetch a solid baseline in the background, then start refreshing.
go c.seedRandomPoolBaseline()
}
func (c *Client) markRandomPoolInitialized() bool {
c.poolMu.Lock()
defer c.poolMu.Unlock()
if c.poolInitialized {
c.poolMu.Unlock()
return nil
return false
}
c.poolInitialized = true
c.poolMu.Unlock()
return true
}
// 1. Try to load all cached anime from the database
func (c *Client) loadCachedRandomPool(ctx context.Context) {
cachedJSONs, err := c.db.GetAllCachedAnime(ctx)
if err == nil && len(cachedJSONs) > 0 {
var loadedAnimes []Anime
for _, dataStr := range cachedJSONs {
var anime Anime
if err := json.Unmarshal([]byte(dataStr), &anime); err == nil && anime.MalID > 0 {
loadedAnimes = append(loadedAnimes, anime)
}
if err != nil || len(cachedJSONs) == 0 {
return
}
loadedAnimes := decodeCachedAnime(cachedJSONs)
if len(loadedAnimes) == 0 {
return
}
if len(loadedAnimes) > 0 {
c.poolMu.Lock()
c.randomPool = append(c.randomPool, loadedAnimes...)
c.poolMu.Unlock()
}
func decodeCachedAnime(cachedJSONs []string) []Anime {
loadedAnimes := make([]Anime, 0, len(cachedJSONs))
for _, dataStr := range cachedJSONs {
var anime Anime
if err := json.Unmarshal([]byte(dataStr), &anime); err != nil || anime.MalID == 0 {
continue
}
// 2. Fetch Top Anime page 1 & 2 to ensure we have a robust baseline of high-quality popular anime
go func() {
loadedAnimes = append(loadedAnimes, anime)
}
return loadedAnimes
}
func (c *Client) seedRandomPoolBaseline() {
bgCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
var fetchedAnimes []Anime
top, err := c.GetTopAnime(bgCtx, 1)
if err == nil && len(top.Animes) > 0 {
fetchedAnimes = append(fetchedAnimes, top.Animes...)
}
top2, err := c.GetTopAnime(bgCtx, 2)
if err == nil && len(top2.Animes) > 0 {
fetchedAnimes = append(fetchedAnimes, top2.Animes...)
}
now, err := c.GetSeasonsNow(bgCtx, 1)
if err == nil && len(now.Animes) > 0 {
fetchedAnimes = append(fetchedAnimes, now.Animes...)
}
fetchedAnimes := c.fetchBaselineAnime(bgCtx)
if len(fetchedAnimes) > 0 {
c.poolMu.Lock()
// Use map to de-duplicate any anime
seen := make(map[int]bool)
for _, a := range c.randomPool {
seen[a.MalID] = true
}
for _, a := range fetchedAnimes {
if !seen[a.MalID] {
c.randomPool = append(c.randomPool, a)
seen[a.MalID] = true
}
}
c.poolMu.Unlock()
c.appendUniqueRandomPool(fetchedAnimes)
}
// Start background refresher once seeding completes
c.startPoolRefresher()
}()
}
func (c *Client) fetchBaselineAnime(ctx context.Context) []Anime {
topPageOne := c.fetchTopAnimePage(ctx, 1)
topPageTwo := c.fetchTopAnimePage(ctx, 2)
currentSeason := c.fetchCurrentSeasonAnime(ctx)
fetchedAnimes := make([]Anime, 0, len(topPageOne)+len(topPageTwo)+len(currentSeason))
fetchedAnimes = append(fetchedAnimes, topPageOne...)
fetchedAnimes = append(fetchedAnimes, topPageTwo...)
fetchedAnimes = append(fetchedAnimes, currentSeason...)
return fetchedAnimes
}
func (c *Client) fetchTopAnimePage(ctx context.Context, page int) []Anime {
top, err := c.GetTopAnime(ctx, page)
if err != nil {
return nil
}
return top.Animes
}
func (c *Client) fetchCurrentSeasonAnime(ctx context.Context) []Anime {
now, err := c.GetSeasonsNow(ctx, 1)
if err != nil {
return nil
}
return now.Animes
}
func (c *Client) appendUniqueRandomPool(animes []Anime) {
c.poolMu.Lock()
defer c.poolMu.Unlock()
seen := make(map[int]bool, len(c.randomPool)+len(animes))
for _, anime := range c.randomPool {
seen[anime.MalID] = true
}
for _, anime := range animes {
if seen[anime.MalID] {
continue
}
c.randomPool = append(c.randomPool, anime)
seen[anime.MalID] = true
}
}
// startPoolRefresher runs in the background to slowly mix in true random anime
func (c *Client) startPoolRefresher() {
ticker := time.NewTicker(30 * time.Second)
@@ -162,7 +209,7 @@ func (c *Client) GetRandomAnime(ctx context.Context) (Anime, error) {
c.poolMu.Unlock()
if !initialized {
_ = c.seedRandomPool(ctx)
c.seedRandomPool(ctx)
}
c.poolMu.RLock()

64
integrations/jikan/types.go
View File

@@ -33,12 +33,18 @@ type Aired struct {
String string `json:"string"`
}
type TitleEntry struct {
Type string `json:"type"`
Title string `json:"title"`
}
type Anime struct {
MalID int `json:"mal_id"`
Title string `json:"title"`
TitleEnglish string `json:"title_english"`
TitleJapanese string `json:"title_japanese"`
TitleSynonyms []string `json:"title_synonyms"`
Titles []TitleEntry `json:"titles"`
Images struct {
Jpg struct {
LargeImageURL string `json:"large_image_url"`
@@ -230,35 +236,34 @@ func (a Anime) DurationSeconds() float64 {
return 0
}
var hours, minutes int
var isHours bool
var currentNum string
var currentValue int
hasValue := false
for _, c := range a.Duration {
if c >= '0' && c <= '9' {
currentNum += string(c)
} else if c == ' ' && currentNum != "" {
val, _ := strconv.Atoi(currentNum)
if isHours {
hours = val
} else {
minutes = val
for _, token := range strings.Fields(strings.ToLower(a.Duration)) {
value, err := strconv.Atoi(token)
if err == nil {
currentValue = value
hasValue = true
continue
}
currentNum = ""
} else if len(currentNum) > 0 && (c == 'h' || c == 'H') {
isHours = true
val, _ := strconv.Atoi(currentNum)
hours = val
currentNum = ""
if !hasValue {
continue
}
switch {
case strings.HasPrefix(token, "h"):
hours = currentValue
hasValue = false
case strings.HasPrefix(token, "m"):
minutes = currentValue
hasValue = false
}
}
if currentNum != "" {
val, _ := strconv.Atoi(currentNum)
if isHours {
hours = val
} else {
minutes = val
}
if hasValue {
minutes = currentValue
}
return float64(hours*60+minutes) * 60
}
@@ -455,13 +460,16 @@ type ReviewsResponse struct {
Pagination Pagination `json:"pagination"`
}
// DisplayTitle returns English title if available, otherwise Japanese, then default.
// DisplayTitle returns English title if available, otherwise default title, titles[0], then Japanese.
func (a Anime) DisplayTitle() string {
if a.TitleEnglish != "" {
return a.TitleEnglish
}
if a.TitleJapanese != "" {
return a.TitleJapanese
}
if a.Title != "" {
return a.Title
}
if len(a.Titles) > 0 && a.Titles[0].Title != "" {
return a.Titles[0].Title
}
return a.TitleJapanese
}

27
integrations/jikan/types_test.go Normal file
View File

@@ -0,0 +1,27 @@
package jikan
import "testing"
func TestAnimeDisplayTitlePrefersTitleBeforeJapanese(t *testing.T) {
anime := Anime{
Title: "Cyberpunk: Edgerunners",
TitleJapanese: "サイバーパンク エッジランナーズ",
}
if got := anime.DisplayTitle(); got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want default title", got)
}
}
func TestAnimeDisplayTitleFallsBackToFirstTitleEntryBeforeJapanese(t *testing.T) {
anime := Anime{
TitleJapanese: "サイバーパンク エッジランナーズ",
Titles: []TitleEntry{
{Type: "Default", Title: "Cyberpunk: Edgerunners"},
},
}
if got := anime.DisplayTitle(); got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want first title entry", got)
}
}

102
integrations/playback/allanime/availability.go Normal file
View File

@@ -0,0 +1,102 @@
package allanime
import (
"context"
"fmt"
"mal/internal/domain"
"strconv"
"strings"
)
type AvailableEpisodes struct {
Sub []string
Dub []string
Raw []string
}
func (c *AllAnimeProvider) GetEpisodeAvailability(ctx context.Context, animeID int, titleCandidates []string) (domain.EpisodeAvailability, error) {
showID, err := c.ResolveEpisodeProviderID(ctx, animeID, titleCandidates)
if err != nil {
return domain.EpisodeAvailability{}, err
}
return c.GetEpisodeAvailabilityByProviderID(ctx, showID)
}
func (c *AllAnimeProvider) GetEpisodeAvailabilityByProviderID(ctx context.Context, showID string) (domain.EpisodeAvailability, error) {
available, err := c.GetAvailableEpisodes(ctx, showID)
if err != nil {
return domain.EpisodeAvailability{}, err
}
sub := parseEpisodeNumbers(append(available.Sub, available.Raw...))
dub := parseEpisodeNumbers(available.Dub)
return domain.EpisodeAvailability{Sub: sub, Dub: dub}, nil
}
func (c *AllAnimeProvider) GetAvailableEpisodes(ctx context.Context, showID string) (AvailableEpisodes, error) {
graphqlQuery := `query($showId: String!) {
show(_id: $showId) {
availableEpisodesDetail
lastEpisodeInfo
}
}`
result, err := c.graphqlRequest(ctx, graphqlQuery, map[string]any{"showId": showID})
if err != nil {
return AvailableEpisodes{}, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid response")
}
show, ok := data["show"].(map[string]any)
if !ok || show == nil {
return AvailableEpisodes{}, fmt.Errorf("show not found")
}
detail, ok := show["availableEpisodesDetail"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid detail")
}
return AvailableEpisodes{
Sub: stringSliceFromAny(detail["sub"]),
Dub: stringSliceFromAny(detail["dub"]),
Raw: stringSliceFromAny(detail["raw"]),
}, nil
}
func parseEpisodeNumbers(raw []string) []int {
seen := make(map[int]bool, len(raw))
out := make([]int, 0, len(raw))
for _, value := range raw {
n, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil || n <= 0 || seen[n] {
continue
}
seen[n] = true
out = append(out, n)
}
return out
}
func stringSliceFromAny(value any) []string {
items, ok := value.([]any)
if !ok {
return nil
}
values := make([]string, 0, len(items))
for _, item := range items {
str, ok := item.(string)
if !ok {
continue
}
values = append(values, str)
}
return values
}

690
integrations/playback/allanime/client.go
View File

@@ -1,49 +1,26 @@
// Package allanime provides an integration with the AllAnime API for episode playback.
package allanime
import (
"bytes"
"context"
"crypto/aes"
"crypto/cipher"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"mal/internal/domain"
"mal/pkg"
netutil "mal/pkg/net"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
const (
allAnimeBaseURL = "https://api.allanime.day"
allAnimeReferer = "https://allmanga.to/"
allAnimeSiteURL = "https://allanime.day"
allAnimeReferer = "https://youtu-chan.com"
allAnimeOrigin = "https://youtu-chan.com"
defaultUserAgent = netutil.Firefox121
)
var (
aesKeys = []string{"Xot36i3lK3:v1", "SimtVuagFbGR2K7P"}
)
type searchResult struct {
ID string
MalID string
Name string
}
type AvailableEpisodes struct {
Sub []string
Dub []string
Raw []string
}
type AllAnimeProvider struct {
httpClient *http.Client
utlsClient *http.Client
@@ -67,139 +44,23 @@ func (c *AllAnimeProvider) Name() string {
return "AllAnime"
}
const searchQuery = `query(
$search: SearchInput
$translationType: VaildTranslationTypeEnumType
$limit: Int = 40
$page: Int = 1
$countryOrigin: VaildCountryOriginEnumType = ALL
) {
shows(
search: $search
limit: $limit
page: $page
translationType: $translationType
countryOrigin: $countryOrigin
) {
edges {
_id
malId
name
}
}
}`
func (c *AllAnimeProvider) Search(ctx context.Context, query string, mode string) ([]searchResult, error) {
type searchData struct {
Shows struct {
Edges []struct {
ID string `json:"_id"`
MalID string `json:"malId"`
Name string `json:"name"`
} `json:"edges"`
} `json:"shows"`
}
type searchInput struct {
AllowAdult bool `json:"allowAdult"`
AllowUnknown bool `json:"allowUnknown"`
Query string `json:"query"`
}
type searchVariables struct {
Search searchInput `json:"search"`
TranslationType string `json:"translationType"`
}
vars := searchVariables{
Search: searchInput{
AllowAdult: false,
AllowUnknown: false,
Query: query,
},
TranslationType: mode,
}
data, err := graphql.Post[searchData](ctx, c.httpClient, allAnimeBaseURL+"/api", searchQuery, vars, graphql.PostOptions{
Headers: map[string]string{
"Referer": allAnimeReferer,
"User-Agent": defaultUserAgent,
},
BodyMax: netutil.MiB2,
})
if err != nil {
return nil, err
}
out := make([]searchResult, 0, len(data.Shows.Edges))
for _, edge := range data.Shows.Edges {
id := edge.ID
malID := edge.MalID
name := edge.Name
if unquoted, err := strconv.Unquote("\"" + name + "\""); err == nil {
name = unquoted
}
name = strings.TrimSpace(name)
if id == "" {
continue
}
out = append(out, searchResult{ID: id, MalID: malID, Name: name})
}
return out, nil
}
func (c *AllAnimeProvider) GetStreams(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string) (*domain.StreamResult, error) {
// 1. Search for the show to get its AllAnime ID
// Try each title candidate, preferring results with matching malId
targetMalIDStr := strconv.Itoa(animeID)
var showID string
var firstAvailableShowID string
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil || len(searchResults) == 0 {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
showID = res.ID
break
}
}
if showID != "" {
break
}
if firstAvailableShowID == "" {
firstAvailableShowID = searchResults[0].ID
}
}
if showID == "" {
showID = firstAvailableShowID
}
showID := c.resolveShowIDWithFallback(ctx, animeID, titleCandidates, mode)
if showID == "" {
return nil, fmt.Errorf("allanime: show not found for malID %d", animeID)
}
// 2. Get sources
sources, err := c.GetEpisodeSources(ctx, showID, episode, mode)
if err != nil || len(sources) == 0 {
return nil, fmt.Errorf("allanime: no sources for show %s", showID)
}
// 3. Return the first usable source
primary := sources[0]
result := &domain.StreamResult{
URL: primary.URL,
Referer: primary.Referer,
Type: primary.Type,
}
for _, sub := range primary.Subtitles {
@@ -212,65 +73,6 @@ func (c *AllAnimeProvider) GetStreams(ctx context.Context, animeID int, titleCan
return result, nil
}
func (c *AllAnimeProvider) GetEpisodeAvailability(ctx context.Context, animeID int, titleCandidates []string) (domain.EpisodeAvailability, error) {
showID, err := c.ResolveEpisodeProviderID(ctx, animeID, titleCandidates)
if err != nil {
return domain.EpisodeAvailability{}, err
}
return c.GetEpisodeAvailabilityByProviderID(ctx, showID)
}
func (c *AllAnimeProvider) ResolveEpisodeProviderID(ctx context.Context, animeID int, titleCandidates []string) (string, error) {
for _, mode := range []string{"sub", "dub"} {
showID, err := c.resolveShowIDStrict(ctx, animeID, titleCandidates, mode)
if err == nil {
return showID, nil
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d", animeID)
}
func (c *AllAnimeProvider) GetEpisodeAvailabilityByProviderID(ctx context.Context, showID string) (domain.EpisodeAvailability, error) {
available, err := c.GetAvailableEpisodes(ctx, showID)
if err != nil {
return domain.EpisodeAvailability{}, err
}
sub := parseEpisodeNumbers(append(available.Sub, available.Raw...))
dub := parseEpisodeNumbers(available.Dub)
return domain.EpisodeAvailability{Sub: sub, Dub: dub}, nil
}
func (c *AllAnimeProvider) resolveShowIDStrict(ctx context.Context, animeID int, titleCandidates []string, mode string) (string, error) {
targetMalIDStr := strconv.Itoa(animeID)
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
return res.ID, nil
}
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d in %s search", animeID, mode)
}
func parseEpisodeNumbers(raw []string) []int {
seen := make(map[int]bool, len(raw))
out := make([]int, 0, len(raw))
for _, value := range raw {
n, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil || n <= 0 || seen[n] {
continue
}
seen[n] = true
out = append(out, n)
}
return out
}
func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, variables map[string]any) (map[string]any, error) {
if mode, ok := variables["translationType"].(string); ok {
variables["translationType"] = strings.ToLower(mode)
@@ -295,13 +97,13 @@ func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, var
req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("User-Agent", defaultUserAgent)
resp, respBody, err := executeAndReadResponse(c.httpClient, req, "execute graphql request", "read graphql response")
statusCode, respBody, err := executeAndReadResponse(c.httpClient, req, "execute graphql request", "read graphql response")
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("graphql status %d", resp.StatusCode)
if statusCode != http.StatusOK {
return nil, fmt.Errorf("graphql status %d", statusCode)
}
var parsed map[string]any
@@ -316,487 +118,17 @@ func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, var
return parsed, nil
}
const episodeQueryHash = "d405d0edd690624b66baba3068e0edc3ac90f1597d898a1ec8db4e5c43c00fec"
func (c *AllAnimeProvider) graphqlRequestWithHash(ctx context.Context, showID, episode, mode string) (map[string]any, error) {
mode = strings.ToLower(mode)
varsJSON := fmt.Sprintf(`{"showId":"%s","translationType":"%s","episodeString":"%s"}`, showID, mode, episode)
extJSON := fmt.Sprintf(`{"persistedQuery":{"version":1,"sha256Hash":"%s"}}`, episodeQueryHash)
apiURL := fmt.Sprintf("%s/api?variables=%s&extensions=%s",
allAnimeBaseURL,
url.QueryEscape(varsJSON),
url.QueryEscape(extJSON))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, apiURL, nil)
if err != nil {
return nil, fmt.Errorf("create GET request: %w", err)
}
req.Header.Set("User-Agent", defaultUserAgent)
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Language", "en-US,en;q=0.5")
req.Header.Set("Accept-Encoding", "identity")
req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("Origin", allAnimeOrigin)
req.Header.Set("Sec-Fetch-Dest", "empty")
req.Header.Set("Sec-Fetch-Mode", "cors")
req.Header.Set("Sec-Fetch-Site", "cross-site")
resp, respBody, err := executeAndReadResponse(c.utlsClient, req, "execute GET request", "read response")
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("GET status %d: %s", resp.StatusCode, string(respBody))
}
var parsed map[string]any
if err := json.Unmarshal(respBody, &parsed); err != nil {
return nil, fmt.Errorf("decode response: %w", err)
}
if errs, ok := parsed["errors"].([]any); ok && len(errs) > 0 {
return nil, fmt.Errorf("graphql error: %v", errs[0])
}
data, ok := parsed["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("no data in response")
}
var toBeParsed string
if s, ok := data["tobeparsed"].(string); ok && s != "" {
toBeParsed = s
} else if episodeData, ok := data["episode"].(map[string]any); ok {
if s, ok := episodeData["tobeparsed"].(string); ok {
toBeParsed = s
}
}
if toBeParsed != "" {
decrypted, err := decryptTobeparsed(toBeParsed)
if err != nil {
return nil, fmt.Errorf("decrypt tobeparsed: %w", err)
}
var ep map[string]any
if jerr := json.Unmarshal(decrypted, &ep); jerr != nil {
return nil, fmt.Errorf("unmarshal decrypted: %w", jerr)
}
var sourceURLs []any
if srcs, ok := ep["sourceUrls"].([]any); ok {
sourceURLs = srcs
} else if epInner, ok := ep["episode"].(map[string]any); ok {
if srcs, ok := epInner["sourceUrls"].([]any); ok {
sourceURLs = srcs
}
}
if len(sourceURLs) > 0 {
return map[string]any{
"episode": map[string]any{
"sourceUrls": sourceURLs,
},
}, nil
}
}
if episodeData, ok := data["episode"].(map[string]any); ok {
if srcs, ok := episodeData["sourceUrls"].([]any); ok && len(srcs) > 0 {
return parsed, nil
}
}
return nil, fmt.Errorf("no usable data in response")
}
// GetEpisodeSources fetches stream URLs for a given show, episode, and mode (dub/sub).
func (c *AllAnimeProvider) GetEpisodeSources(ctx context.Context, showID string, episode string, mode string) ([]StreamSource, error) {
episodeQuery := `query($showId: String!, $translationType: VaildTranslationTypeEnumType!, $episodeString: String!) {
episode(showId: $showId, translationType: $translationType, episodeString: $episodeString) {
sourceUrls
}
}`
result, err := c.graphqlRequestWithHash(ctx, showID, episode, mode)
if err == nil {
sources := c.extractSourceURLsFromData(ctx, result)
if len(sources) > 0 {
return sources, nil
}
}
result, err = c.graphqlRequest(ctx, episodeQuery, map[string]any{
"showId": showID,
"translationType": mode,
"episodeString": episode,
})
if err != nil {
return nil, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid source response")
}
rawSourceURLs, ok := data["episode"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid episode response")
}
sourceURLs, ok := rawSourceURLs["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil, fmt.Errorf("no source urls")
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil, fmt.Errorf("no source references")
}
out := c.resolveSourceReferences(ctx, references)
if len(out) == 0 {
return nil, fmt.Errorf("no playable sources extracted")
}
return out, nil
}
func (c *AllAnimeProvider) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource {
episodeData, ok := data["episode"].(map[string]any)
if !ok {
return nil
}
sourceURLs, ok := episodeData["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil
}
return c.resolveSourceReferences(ctx, references)
}
func (c *AllAnimeProvider) resolveSourceReferences(ctx context.Context, references []sourceReference) []StreamSource {
out := make([]StreamSource, 0, len(references))
for _, ref := range references {
target := strings.TrimSpace(ref.URL)
if target == "" {
continue
}
if strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") {
sourceType := detectStreamType(target)
if sourceType == "unknown" {
sourceType = detectEmbedType(target)
}
out = append(out, buildStreamSource(target, sourceType, ref.Name))
continue
}
decoded := decodeSourceURL(target)
if decoded == "" {
continue
}
if strings.HasPrefix(decoded, "http://") || strings.HasPrefix(decoded, "https://") {
sourceType := detectStreamType(decoded)
if sourceType == "unknown" {
sourceType = detectEmbedType(decoded)
}
out = append(out, buildStreamSource(decoded, sourceType, ref.Name))
continue
}
if !strings.HasPrefix(decoded, "/") {
decoded = "/" + decoded
}
extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded)
if err != nil {
continue
}
out = append(out, extracted...)
}
return out
}
func executeAndReadResponse(client *http.Client, req *http.Request, executeErrPrefix string, readErrPrefix string) (*http.Response, []byte, error) {
func executeAndReadResponse(client *http.Client, req *http.Request, executeErrPrefix string, readErrPrefix string) (int, []byte, error) {
resp, err := client.Do(req)
if err != nil {
return nil, nil, fmt.Errorf("%s: %w", executeErrPrefix, err)
return 0, nil, fmt.Errorf("%s: %w", executeErrPrefix, err)
}
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil {
return nil, nil, fmt.Errorf("%s: %w", readErrPrefix, err)
return 0, nil, fmt.Errorf("%s: %w", readErrPrefix, err)
}
return resp, body, nil
}
func buildStreamSource(url, sourceType, provider string) StreamSource {
return StreamSource{
URL: url,
Provider: provider,
Type: sourceType,
Referer: allAnimeReferer,
}
}
type sourceReference struct {
URL string
Name string
}
// buildSourceReferences orders source URLs by provider priority, deduplicating entries.
func buildSourceReferences(rawSourceURLs []any) []sourceReference {
priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"}
prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}}
prioritized := make(map[string]sourceReference)
fallback := make([]sourceReference, 0, len(rawSourceURLs))
seen := make(map[string]struct{})
for _, source := range rawSourceURLs {
item, ok := source.(map[string]any)
if !ok {
continue
}
sourceURL, _ := item["sourceUrl"].(string)
sourceName, _ := item["sourceName"].(string)
sourceURL = strings.TrimSpace(sourceURL)
sourceName = strings.TrimSpace(sourceName)
if sourceURL == "" {
continue
}
if _, exists := seen[sourceURL]; exists {
continue
}
seen[sourceURL] = struct{}{}
ref := sourceReference{URL: sourceURL, Name: sourceName}
normalized := strings.ToLower(sourceName)
// separate prioritized providers from fallback
if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider {
if _, exists := prioritized[normalized]; !exists {
prioritized[normalized] = ref
}
continue
}
fallback = append(fallback, ref)
}
// output: prioritized in order, then fallback
ordered := make([]sourceReference, 0, len(prioritized)+len(fallback))
for _, provider := range priorityOrder {
if ref, ok := prioritized[provider]; ok {
ordered = append(ordered, ref)
}
}
ordered = append(ordered, fallback...)
return ordered
}
func decryptTobeparsed(encoded string) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
return nil, fmt.Errorf("base64 decode failed: %w", err)
}
if len(raw) < 29 {
return nil, fmt.Errorf("encrypted payload too short")
}
version := raw[0]
iv := raw[1:13]
cipherText := raw[13 : len(raw)-16]
for _, keyStr := range aesKeys {
key := sha256.Sum256([]byte(keyStr))
block, err := aes.NewCipher(key[:])
if err != nil {
continue
}
if version == 1 {
plainText := tryDecryptCTR(block, iv, cipherText)
if json.Valid(plainText) {
return plainText, nil
}
}
gcm, err := cipher.NewGCM(block)
if err == nil {
tag := raw[len(raw)-16:]
combined := append(append([]byte{}, cipherText...), tag...)
plainText, openErr := gcm.Open(nil, iv, combined, nil)
if openErr == nil && json.Valid(plainText) {
return plainText, nil
}
}
}
return nil, fmt.Errorf("decryption failed")
}
func tryDecryptCTR(block cipher.Block, iv []byte, cipherText []byte) []byte {
ctrIV := append([]byte{}, iv...)
ctrIV = append(ctrIV, 0x00, 0x00, 0x00, 0x02)
ctr := cipher.NewCTR(block, ctrIV)
plainText := make([]byte, len(cipherText))
ctr.XORKeyStream(plainText, cipherText)
return plainText
}
// GetAvailableEpisodes returns the count of sub/dub/raw episodes available for a show.
func (c *AllAnimeProvider) GetAvailableEpisodes(ctx context.Context, showID string) (AvailableEpisodes, error) {
graphqlQuery := `query($showId: String!) {
show(_id: $showId) {
availableEpisodesDetail
lastEpisodeInfo
}
}`
result, err := c.graphqlRequest(ctx, graphqlQuery, map[string]any{"showId": showID})
if err != nil {
return AvailableEpisodes{}, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid response")
}
show, ok := data["show"].(map[string]any)
if !ok || show == nil {
return AvailableEpisodes{}, fmt.Errorf("show not found")
}
detail, ok := show["availableEpisodesDetail"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid detail")
}
var count AvailableEpisodes
if sub, ok := detail["sub"].([]any); ok {
for _, s := range sub {
if str, ok := s.(string); ok {
count.Sub = append(count.Sub, str)
}
}
}
if dub, ok := detail["dub"].([]any); ok {
for _, s := range dub {
if str, ok := s.(string); ok {
count.Dub = append(count.Dub, str)
}
}
}
if raw, ok := detail["raw"].([]any); ok {
for _, s := range raw {
if str, ok := s.(string); ok {
count.Raw = append(count.Raw, str)
}
}
}
return count, nil
}
func decodeSourceURL(encoded string) string {
if encoded == "" {
return ""
}
encoded = strings.TrimPrefix(encoded, "--")
substitutions := map[string]string{
"79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E",
"7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J",
"73": "K", "74": "L", "75": "M", "76": "N", "77": "O",
"68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T",
"6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y",
"62": "Z",
"59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e",
"5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j",
"53": "k", "54": "l", "55": "m", "56": "n", "57": "o",
"48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t",
"4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y",
"42": "z",
"08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4",
"0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9",
"15": "-", "16": ".", "67": "_", "46": "~", "02": ":",
"17": "/", "07": "?", "1b": "#", "63": "[", "65": "]",
"78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(",
"11": ")", "12": "*", "13": "+", "14": ",", "03": ";",
"05": "=", "1d": "%",
}
var result strings.Builder
for idx := 0; idx < len(encoded); {
if idx+2 <= len(encoded) {
pair := encoded[idx : idx+2]
if sub, ok := substitutions[pair]; ok {
result.WriteString(sub)
idx += 2
continue
}
}
result.WriteByte(encoded[idx])
idx++
}
decoded := result.String()
if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") {
decoded = strings.Replace(decoded, "/clock", "/clock.json", 1)
}
return decoded
}
func detectStreamType(sourceURL string) string {
lower := strings.ToLower(sourceURL)
if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") {
return "m3u8"
}
if strings.Contains(lower, ".mp4") {
return "mp4"
}
return "unknown"
}
func detectEmbedType(rawURL string) string {
lower := strings.ToLower(rawURL)
embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"}
for _, host := range embedHosts {
if strings.Contains(lower, host) {
return "embed"
}
}
return "unknown"
return resp.StatusCode, body, nil
}

268
integrations/playback/allanime/client_test.go
View File

@@ -20,167 +20,182 @@ func isLikelyMP4(data []byte) bool {
return string(data[4:8]) == "ftyp"
}
func TestDecodeSourceURL(t *testing.T) {
t.Parallel()
tests := []struct {
type stringTransformTestCase struct {
name string
encoded string
input string
want string
}{
{
name: "empty returns empty",
encoded: "",
want: "",
},
{
name: "with double prefix stripped",
encoded: "--example.com/video.mp4",
want: "example.com/video.mp4",
},
{
name: "hex substitution",
encoded: "7aexample",
want: "Bexample",
},
{
name: "mixed substitution",
encoded: "79url7a01",
want: "AurlB9",
},
{
name: "clock replacement",
encoded: "/clock",
want: "/clock.json",
},
{
name: "no clock replacement if already json",
encoded: "/clock.json",
want: "/clock.json",
},
{
name: "complex url",
encoded: "--79stream7acom",
want: "AstreamBcom",
},
}
type sourceReferencesTestCase struct {
name string
rawURLs []any
wantRefs []sourceReference
}
func runStringTransformTests(t *testing.T, tests []stringTransformTestCase, fn func(string) string) {
t.Helper()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := decodeSourceURL(tt.encoded)
got := fn(tt.input)
if got != tt.want {
t.Errorf("decodeSourceURL(%q) = %q, want %q", tt.encoded, got, tt.want)
t.Errorf("got %q for input %q, want %q", got, tt.input, tt.want)
}
})
}
}
func runSourceReferenceTests(t *testing.T, tests []sourceReferencesTestCase) {
t.Helper()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := buildSourceReferences(tt.rawURLs)
if len(got) != len(tt.wantRefs) {
t.Errorf("got %d refs, want %d", len(got), len(tt.wantRefs))
return
}
for i, want := range tt.wantRefs {
if got[i].URL != want.URL {
t.Errorf("ref[%d].URL = %q, want %q", i, got[i].URL, want.URL)
}
if got[i].Name != want.Name {
t.Errorf("ref[%d].Name = %q, want %q", i, got[i].Name, want.Name)
}
}
})
}
}
func TestDecodeSourceURL(t *testing.T) {
t.Parallel()
tests := []stringTransformTestCase{
{
name: "empty returns empty",
input: "",
want: "",
},
{
name: "with double prefix stripped",
input: "--example.com/video.mp4",
want: "example.com/video.mp4",
},
{
name: "hex substitution",
input: "7aexample",
want: "Bexample",
},
{
name: "mixed substitution",
input: "79url7a01",
want: "AurlB9",
},
{
name: "clock replacement",
input: "/clock",
want: "/clock.json",
},
{
name: "no clock replacement if already json",
input: "/clock.json",
want: "/clock.json",
},
{
name: "complex url",
input: "--79stream7acom",
want: "AstreamBcom",
},
}
runStringTransformTests(t, tests, decodeSourceURL)
}
func TestDetectStreamType(t *testing.T) {
t.Parallel()
tests := []struct {
name string
url string
wantType string
}{
tests := []stringTransformTestCase{
{
name: "m3u8 extension",
url: "https://example.com/video.m3u8",
wantType: "m3u8",
input: "https://example.com/video.m3u8",
want: "m3u8",
},
{
name: "master m3u8",
url: "https://example.com/master.m3u8",
wantType: "m3u8",
input: "https://example.com/master.m3u8",
want: "m3u8",
},
{
name: "mp4 extension",
url: "https://example.com/video.mp4",
wantType: "mp4",
input: "https://example.com/video.mp4",
want: "mp4",
},
{
name: "unknown",
url: "https://example.com/video.avi",
wantType: "unknown",
input: "https://example.com/video.avi",
want: "unknown",
},
{
name: "empty returns unknown",
url: "",
wantType: "unknown",
input: "",
want: "unknown",
},
{
name: "case insensitive - M3U8",
url: "https://example.com/MASTER.M3U8",
wantType: "m3u8",
input: "https://example.com/MASTER.M3U8",
want: "m3u8",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := detectStreamType(tt.url)
if got != tt.wantType {
t.Errorf("detectStreamType(%q) = %q, want %q", tt.url, got, tt.wantType)
}
})
}
runStringTransformTests(t, tests, detectStreamType)
}
func TestDetectEmbedType(t *testing.T) {
t.Parallel()
tests := []struct {
name string
url string
wantType string
}{
tests := []stringTransformTestCase{
{
name: "streamwish",
url: "https://streamwish.com/e/abc123",
wantType: "embed",
input: "https://streamwish.com/e/abc123",
want: "embed",
},
{
name: "streamsb",
url: "https://streamsb.com/e/abc123",
wantType: "embed",
input: "https://streamsb.com/e/abc123",
want: "embed",
},
{
name: "mp4upload",
url: "https://mp4upload.com/e/abc123",
wantType: "embed",
input: "https://mp4upload.com/e/abc123",
want: "embed",
},
{
name: "ok.ru",
url: "https://ok.ru/video/123",
wantType: "embed",
input: "https://ok.ru/video/123",
want: "embed",
},
{
name: "gogoplay",
url: "https://gogoplay.io/embed/123",
wantType: "embed",
input: "https://gogoplay.io/embed/123",
want: "embed",
},
{
name: "streamlare",
url: "https://streamlare.com/e/abc",
wantType: "embed",
input: "https://streamlare.com/e/abc",
want: "embed",
},
{
name: "unknown host",
url: "https://unknown.com/video",
wantType: "unknown",
input: "https://unknown.com/video",
want: "unknown",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := detectEmbedType(tt.url)
if got != tt.wantType {
t.Errorf("detectEmbedType(%q) = %q, want %q", tt.url, got, tt.wantType)
}
})
}
runStringTransformTests(t, tests, detectEmbedType)
}
func TestBuildStreamSource(t *testing.T) {
@@ -204,14 +219,21 @@ func TestBuildStreamSource(t *testing.T) {
})
}
func TestResolveDirectSourceSkipsEmbeds(t *testing.T) {
t.Parallel()
if _, ok := resolveDirectSource(sourceReference{
URL: "https://ok.ru/videoembed/123",
Name: "ok",
}); ok {
t.Fatal("expected embed URL to require extraction")
}
}
func TestBuildSourceReferences(t *testing.T) {
t.Parallel()
tests := []struct {
name string
rawURLs []any
wantRefs []sourceReference
}{
tests := []sourceReferencesTestCase{
{
name: "empty returns empty",
rawURLs: nil,
@@ -263,26 +285,7 @@ func TestBuildSourceReferences(t *testing.T) {
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := buildSourceReferences(tt.rawURLs)
if len(got) != len(tt.wantRefs) {
t.Errorf("got %d refs, want %d", len(got), len(tt.wantRefs))
return
}
for i, want := range tt.wantRefs {
if got[i].URL != want.URL {
t.Errorf("ref[%d].URL = %q, want %q", i, got[i].URL, want.URL)
}
if got[i].Name != want.Name {
t.Errorf("ref[%d].Name = %q, want %q", i, got[i].Name, want.Name)
}
}
})
}
runSourceReferenceTests(t, tests)
}
func TestBuildSourceReferencesOrder(t *testing.T) {
@@ -391,6 +394,27 @@ func TestIsLikelyMP4(t *testing.T) {
}
}
func TestParseOKRUSources(t *testing.T) {
t.Parallel()
body := `{"flashvars":{"metadata":"{\"hlsManifestUrl\":\"https://vd.example.test/video.m3u8?cmd=videoPlayerCdn\\u0026id=123\"}"}}`
got := parseOKRUSources(body, allAnimeReferer)
if len(got) != 1 {
t.Fatalf("len(got) = %d, want 1", len(got))
}
if got[0].URL != "https://vd.example.test/video.m3u8?cmd=videoPlayerCdn&id=123" {
t.Fatalf("URL = %q", got[0].URL)
}
if got[0].Type != "m3u8" {
t.Fatalf("Type = %q, want m3u8", got[0].Type)
}
if got[0].Provider != "ok" {
t.Fatalf("Provider = %q, want ok", got[0].Provider)
}
}
func TestDecryptTobeparsed(t *testing.T) {
t.Parallel()

235
integrations/playback/allanime/crypto.go Normal file
View File

@@ -0,0 +1,235 @@
package allanime
import (
"crypto/aes"
"crypto/cipher"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
)
var (
aesKeys = []string{"Xot36i3lK3:v1", "SimtVuagFbGR2K7P"}
)
func decryptTobeparsed(encoded string) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
return nil, fmt.Errorf("base64 decode failed: %w", err)
}
if len(raw) < 29 {
return nil, fmt.Errorf("encrypted payload too short")
}
version := raw[0]
iv := raw[1:13]
cipherText := raw[13 : len(raw)-16]
for _, keyStr := range aesKeys {
key := sha256.Sum256([]byte(keyStr))
block, err := aes.NewCipher(key[:])
if err != nil {
continue
}
if version == 1 {
plainText := tryDecryptCTR(block, iv, cipherText)
if json.Valid(plainText) {
return plainText, nil
}
}
gcm, err := cipher.NewGCM(block)
if err == nil {
tag := raw[len(raw)-16:]
combined := append(append([]byte{}, cipherText...), tag...)
plainText, openErr := gcm.Open(nil, iv, combined, nil)
if openErr == nil && json.Valid(plainText) {
return plainText, nil
}
}
}
return nil, fmt.Errorf("decryption failed")
}
func tryDecryptCTR(block cipher.Block, iv []byte, cipherText []byte) []byte {
ctrIV := append([]byte{}, iv...)
ctrIV = append(ctrIV, 0x00, 0x00, 0x00, 0x02)
ctr := cipher.NewCTR(block, ctrIV)
plainText := make([]byte, len(cipherText))
ctr.XORKeyStream(plainText, cipherText)
return plainText
}
func decodeSourceURL(encoded string) string {
if encoded == "" {
return ""
}
encoded = strings.TrimPrefix(encoded, "--")
substitutions := map[string]string{
"79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E",
"7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J",
"73": "K", "74": "L", "75": "M", "76": "N", "77": "O",
"68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T",
"6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y",
"62": "Z",
"59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e",
"5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j",
"53": "k", "54": "l", "55": "m", "56": "n", "57": "o",
"48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t",
"4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y",
"42": "z",
"08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4",
"0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9",
"15": "-", "16": ".", "67": "_", "46": "~", "02": ":",
"17": "/", "07": "?", "1b": "#", "63": "[", "65": "]",
"78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(",
"11": ")", "12": "*", "13": "+", "14": ",", "03": ";",
"05": "=", "1d": "%",
}
var result strings.Builder
for idx := 0; idx < len(encoded); {
if idx+2 <= len(encoded) {
pair := encoded[idx : idx+2]
if sub, ok := substitutions[pair]; ok {
result.WriteString(sub)
idx += 2
continue
}
}
result.WriteByte(encoded[idx])
idx++
}
decoded := result.String()
if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") {
decoded = strings.Replace(decoded, "/clock", "/clock.json", 1)
}
return decoded
}
func responseFromTobeparsed(data map[string]any) (map[string]any, error) {
toBeParsed := firstNonEmptyString(
nestedString(data, "tobeparsed"),
nestedString(data, "episode", "tobeparsed"),
)
if toBeParsed == "" {
return nil, nil
}
decrypted, err := decryptTobeparsed(toBeParsed)
if err != nil {
return nil, fmt.Errorf("decrypt tobeparsed: %w", err)
}
parsed, err := parseGraphQLResponse(decrypted, "unmarshal decrypted")
if err != nil {
return nil, err
}
sourceURLs := firstNonEmptySlice(
nestedSlice(parsed, "sourceUrls"),
nestedSlice(parsed, "episode", "sourceUrls"),
)
if len(sourceURLs) == 0 {
return nil, nil
}
return map[string]any{
"episode": map[string]any{
"sourceUrls": sourceURLs,
},
}, nil
}
func hasEpisodeSourceURLs(data map[string]any) bool {
return len(nestedSlice(data, "episode", "sourceUrls")) > 0
}
func parseGraphQLResponse(respBody []byte, decodeErrPrefix string) (map[string]any, error) {
var parsed map[string]any
if err := json.Unmarshal(respBody, &parsed); err != nil {
return nil, fmt.Errorf("%s: %w", decodeErrPrefix, err)
}
if errs, ok := parsed["errors"].([]any); ok && len(errs) > 0 {
return nil, fmt.Errorf("graphql error: %v", errs[0])
}
return parsed, nil
}
func firstNonEmptyString(values ...string) string {
for _, value := range values {
if value != "" {
return value
}
}
return ""
}
func firstNonEmptySlice(values ...[]any) []any {
for _, value := range values {
if len(value) > 0 {
return value
}
}
return nil
}
func nestedString(data map[string]any, path ...string) string {
value, ok := nestedValue(data, path...)
if !ok {
return ""
}
str, ok := value.(string)
if !ok {
return ""
}
return str
}
func nestedSlice(data map[string]any, path ...string) []any {
value, ok := nestedValue(data, path...)
if !ok {
return nil
}
slice, ok := value.([]any)
if !ok {
return nil
}
return slice
}
func nestedValue(data map[string]any, path ...string) (any, bool) {
var current any = data
for _, key := range path {
currentMap, ok := current.(map[string]any)
if !ok {
return nil, false
}
current, ok = currentMap[key]
if !ok {
return nil, false
}
}
return current, true
}

382
integrations/playback/allanime/extractor.go
View File

@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
"html"
"io"
netutil "mal/pkg/net"
"net/http"
@@ -19,10 +20,27 @@ type providerExtractor struct {
referer string
}
type providerLinkItem struct {
link string
resolutionStr string
}
type providerHLSItem struct {
url string
hardsubLang string
}
type providerResponseData struct {
referer string
links []providerLinkItem
hls []providerHLSItem
subtitles []Subtitle
}
func newProviderExtractor() *providerExtractor {
return &providerExtractor{
httpClient: &http.Client{Timeout: 30 * time.Second},
baseURL: allAnimeBaseURL,
baseURL: allAnimeSiteURL,
referer: allAnimeReferer,
}
}
@@ -63,65 +81,45 @@ func (e *providerExtractor) ExtractVideoLinks(ctx context.Context, providerPath
return e.parseProviderResponse(ctx, string(body)), nil
}
func (e *providerExtractor) ExtractEmbedVideoLinks(ctx context.Context, rawURL string) ([]StreamSource, error) {
resp, err := doProxiedRequest(ctx, e.httpClient, rawURL, e.referer)
if err != nil {
return nil, fmt.Errorf("fetch embed response: %w", err)
}
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil {
return nil, fmt.Errorf("read embed response: %w", err)
}
return parseExternalEmbedResponse(rawURL, string(body), e.referer), nil
}
// parseProviderResponse extracts stream sources from provider JSON response.
func (e *providerExtractor) parseProviderResponse(ctx context.Context, response string) []StreamSource {
sources := make([]StreamSource, 0)
providerReferer := e.referer
var root any
if err := json.Unmarshal([]byte(response), &root); err != nil {
return []StreamSource{}
}
data := collectProviderResponseData(root, e.referer)
sources := buildProviderLinkSources(data.links, data.referer)
sources = append(sources, e.buildProviderHLSSources(ctx, data.hls, data.referer)...)
attachSubtitles(sources, data.subtitles)
return sources
}
type linkItem struct {
link string
resolutionStr string
}
type hlsItem struct {
url string
hardsubLang string
}
linkItems := make([]linkItem, 0)
hlsItems := make([]hlsItem, 0)
subtitles := make([]Subtitle, 0)
func collectProviderResponseData(root any, fallbackReferer string) providerResponseData {
data := providerResponseData{referer: fallbackReferer}
var walk func(v any)
walk = func(v any) {
switch x := v.(type) {
case map[string]any:
if ref, ok := x["Referer"].(string); ok && strings.TrimSpace(ref) != "" {
providerReferer = strings.TrimSpace(ref)
}
if link, ok := x["link"].(string); ok {
if res, ok := x["resolutionStr"].(string); ok {
linkItems = append(linkItems, linkItem{link: link, resolutionStr: res})
}
}
if u, ok := x["url"].(string); ok {
if lang, ok := x["hardsub_lang"].(string); ok {
hlsItems = append(hlsItems, hlsItem{url: u, hardsubLang: lang})
}
}
if subs, ok := x["subtitles"].([]any); ok {
for _, sub := range subs {
obj, ok := sub.(map[string]any)
if !ok {
continue
}
lang, _ := obj["lang"].(string)
src, _ := obj["src"].(string)
lang = strings.TrimSpace(lang)
src = strings.TrimSpace(src)
if lang == "" || src == "" {
continue
}
subtitles = append(subtitles, Subtitle{Lang: lang, URL: src})
}
}
collectProviderMapData(x, &data)
for _, child := range x {
walk(child)
}
@@ -133,42 +131,98 @@ func (e *providerExtractor) parseProviderResponse(ctx context.Context, response
}
walk(root)
if providerReferer == "" {
providerReferer = e.referer
if data.referer == "" {
data.referer = fallbackReferer
}
for _, item := range linkItems {
return data
}
func collectProviderMapData(node map[string]any, data *providerResponseData) {
if ref, ok := node["Referer"].(string); ok {
if trimmedRef := strings.TrimSpace(ref); trimmedRef != "" {
data.referer = trimmedRef
}
}
if link, ok := node["link"].(string); ok {
if res, ok := node["resolutionStr"].(string); ok {
data.links = append(data.links, providerLinkItem{link: link, resolutionStr: res})
}
}
if url, ok := node["url"].(string); ok {
if lang, ok := node["hardsub_lang"].(string); ok {
data.hls = append(data.hls, providerHLSItem{url: url, hardsubLang: lang})
}
}
if subs, ok := node["subtitles"].([]any); ok {
data.subtitles = append(data.subtitles, parseProviderSubtitles(subs)...)
}
}
func parseProviderSubtitles(items []any) []Subtitle {
subtitles := make([]Subtitle, 0, len(items))
for _, item := range items {
node, ok := item.(map[string]any)
if !ok {
continue
}
lang, _ := node["lang"].(string)
src, _ := node["src"].(string)
lang = strings.TrimSpace(lang)
src = strings.TrimSpace(src)
if lang == "" || src == "" {
continue
}
subtitles = append(subtitles, Subtitle{Lang: lang, URL: src})
}
return subtitles
}
func buildProviderLinkSources(items []providerLinkItem, referer string) []StreamSource {
sources := make([]StreamSource, 0, len(items))
for _, item := range items {
link := strings.TrimSpace(item.link)
if link == "" {
continue
}
quality := strings.TrimSpace(item.resolutionStr)
sourceType := detectStreamType(link)
if sourceType == "unknown" {
sourceType = detectEmbedType(link)
}
sources = append(sources, StreamSource{
URL: link,
Quality: quality,
Quality: strings.TrimSpace(item.resolutionStr),
Provider: "wixmp",
Type: sourceType,
Referer: providerReferer,
Type: detectProviderSourceType(link),
Referer: referer,
})
}
for _, item := range hlsItems {
if strings.TrimSpace(item.url) == "" {
continue
return sources
}
if item.hardsubLang != "en-US" {
func detectProviderSourceType(link string) string {
sourceType := detectStreamType(link)
if sourceType != "unknown" {
return sourceType
}
return detectEmbedType(link)
}
func (e *providerExtractor) buildProviderHLSSources(ctx context.Context, items []providerHLSItem, referer string) []StreamSource {
sources := make([]StreamSource, 0, len(items))
for _, item := range items {
playlistURL, ok := providerPlaylistURL(item)
if !ok {
continue
}
playlistURL := strings.TrimSpace(item.url)
if strings.Contains(playlistURL, "master.m3u8") {
parsed, err := e.parseM3U8(ctx, playlistURL, providerReferer)
parsed, err := e.parseM3U8(ctx, playlistURL, referer)
if err == nil {
sources = append(sources, parsed...)
}
@@ -180,19 +234,32 @@ func (e *providerExtractor) parseProviderResponse(ctx context.Context, response
Quality: "auto",
Provider: "hls",
Type: "m3u8",
Referer: providerReferer,
Referer: referer,
})
}
if len(subtitles) > 0 && len(sources) > 0 {
return sources
}
func providerPlaylistURL(item providerHLSItem) (string, bool) {
playlistURL := strings.TrimSpace(item.url)
if playlistURL == "" || item.hardsubLang != "en-US" {
return "", false
}
return playlistURL, true
}
func attachSubtitles(sources []StreamSource, subtitles []Subtitle) {
if len(subtitles) == 0 || len(sources) == 0 {
return
}
for idx := range sources {
sources[idx].Subtitles = append([]Subtitle(nil), subtitles...)
}
}
return sources
}
// parseM3U8 fetches a master playlist and extracts individual stream URLs with bandwidth-derived quality.
func (e *providerExtractor) parseM3U8(ctx context.Context, masterURL string, referer string) ([]StreamSource, error) {
resp, err := doProxiedRequest(ctx, e.httpClient, masterURL, referer)
@@ -206,60 +273,159 @@ func (e *providerExtractor) parseM3U8(ctx context.Context, masterURL string, ref
return nil, err
}
lines := strings.Split(string(body), "\n")
baseURL := masterURL
if idx := strings.LastIndex(masterURL, "/"); idx >= 0 {
baseURL = masterURL[:idx+1]
return parseM3U8Sources(string(body), masterURL, referer), nil
}
func parseM3U8Sources(body string, masterURL string, referer string) []StreamSource {
lines := strings.Split(body, "\n")
baseURL := playlistBaseURL(masterURL)
bwPattern := regexp.MustCompile(`BANDWIDTH=(\d+)`)
currentBandwidth := 0
sources := make([]StreamSource, 0)
bwPattern := regexp.MustCompile(`BANDWIDTH=(\d+)`)
for _, line := range lines {
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "#EXT-X-STREAM-INF") {
match := bwPattern.FindStringSubmatch(trimmed)
if len(match) >= 2 {
value, convErr := strconv.Atoi(match[1])
if convErr == nil {
currentBandwidth = value
}
}
if bandwidth, ok := parseStreamBandwidth(trimmed, bwPattern); ok {
currentBandwidth = bandwidth
continue
}
// skip empty lines and non-stream lines
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
if shouldSkipM3U8Line(trimmed) {
continue
}
streamURL := trimmed
if !strings.HasPrefix(streamURL, "http://") && !strings.HasPrefix(streamURL, "https://") {
streamURL = baseURL + streamURL
}
quality := "auto"
kbps := currentBandwidth / 1000
switch {
case kbps >= 8000:
quality = "1080p"
case kbps >= 5000:
quality = "720p"
case kbps >= 2500:
quality = "480p"
case kbps > 0:
quality = "360p"
}
sources = append(sources, StreamSource{
URL: streamURL,
Quality: quality,
URL: resolvePlaylistURL(trimmed, baseURL),
Quality: qualityFromBandwidth(currentBandwidth),
Provider: "hls",
Type: "m3u8",
Referer: referer,
})
}
return sources, nil
return sources
}
func playlistBaseURL(masterURL string) string {
if idx := strings.LastIndex(masterURL, "/"); idx >= 0 {
return masterURL[:idx+1]
}
return masterURL
}
func parseStreamBandwidth(line string, bwPattern *regexp.Regexp) (int, bool) {
if !strings.HasPrefix(line, "#EXT-X-STREAM-INF") {
return 0, false
}
match := bwPattern.FindStringSubmatch(line)
if len(match) < 2 {
return 0, true
}
value, err := strconv.Atoi(match[1])
if err != nil {
return 0, true
}
return value, true
}
func shouldSkipM3U8Line(line string) bool {
return line == "" || strings.HasPrefix(line, "#")
}
func resolvePlaylistURL(streamURL string, baseURL string) string {
if strings.HasPrefix(streamURL, "http://") || strings.HasPrefix(streamURL, "https://") {
return streamURL
}
return baseURL + streamURL
}
func qualityFromBandwidth(bandwidth int) string {
kbps := bandwidth / 1000
switch {
case kbps >= 8000:
return "1080p"
case kbps >= 5000:
return "720p"
case kbps >= 2500:
return "480p"
case kbps > 0:
return "360p"
default:
return "auto"
}
}
func parseExternalEmbedResponse(rawURL string, body string, fallbackReferer string) []StreamSource {
switch {
case strings.Contains(strings.ToLower(rawURL), "ok.ru/"):
return parseOKRUSources(body, fallbackReferer)
case strings.Contains(strings.ToLower(rawURL), "mp4upload.com/"):
return parseMP4UploadSources(body, fallbackReferer)
default:
return nil
}
}
func parseOKRUSources(body string, referer string) []StreamSource {
unescapedBody := html.UnescapeString(body)
manifestPattern := regexp.MustCompile(`\\"hlsManifestUrl\\":\\"([^"]+)\\"|"hlsManifestUrl":"([^"]+)"`)
match := manifestPattern.FindStringSubmatch(unescapedBody)
if len(match) < 3 {
return nil
}
playlistURL := decodeEscapedMediaURL(firstNonEmptyString(match[1], match[2]))
if playlistURL == "" {
return nil
}
return []StreamSource{{
URL: playlistURL,
Quality: "auto",
Provider: "ok",
Type: "m3u8",
Referer: referer,
}}
}
func parseMP4UploadSources(body string, referer string) []StreamSource {
srcPattern := regexp.MustCompile(`(?m)src:\s*"([^"]+)"`)
match := srcPattern.FindStringSubmatch(body)
if len(match) < 2 {
return nil
}
mediaURL := decodeEscapedMediaURL(match[1])
if mediaURL == "" {
return nil
}
return []StreamSource{{
URL: mediaURL,
Provider: "mp4upload",
Type: detectProviderSourceType(mediaURL),
Referer: referer,
}}
}
func decodeEscapedMediaURL(raw string) string {
if unquoted, err := strconv.Unquote(`"` + raw + `"`); err == nil {
raw = unquoted
}
replacer := strings.NewReplacer(
`\\u002F`, `/`,
`\\u0026`, "&",
`\/`, `/`,
`\u002F`, `/`,
`\u0026`, "&",
`&amp;`, "&",
)
return strings.TrimSpace(replacer.Replace(raw))
}

156
integrations/playback/allanime/search.go Normal file
View File

@@ -0,0 +1,156 @@
package allanime
import (
"context"
"fmt"
"mal/pkg"
netutil "mal/pkg/net"
"strconv"
"strings"
)
const searchQuery = `query(
$search: SearchInput
$translationType: VaildTranslationTypeEnumType
$limit: Int = 40
$page: Int = 1
$countryOrigin: VaildCountryOriginEnumType = ALL
) {
shows(
search: $search
limit: $limit
page: $page
translationType: $translationType
countryOrigin: $countryOrigin
) {
edges {
_id
malId
name
}
}
}`
type searchResult struct {
ID string
MalID string
Name string
}
func (c *AllAnimeProvider) Search(ctx context.Context, query string, mode string) ([]searchResult, error) {
type searchData struct {
Shows struct {
Edges []struct {
ID string `json:"_id"`
MalID string `json:"malId"`
Name string `json:"name"`
} `json:"edges"`
} `json:"shows"`
}
type searchInput struct {
AllowAdult bool `json:"allowAdult"`
AllowUnknown bool `json:"allowUnknown"`
Query string `json:"query"`
}
type searchVariables struct {
Search searchInput `json:"search"`
TranslationType string `json:"translationType"`
}
vars := searchVariables{
Search: searchInput{
AllowAdult: false,
AllowUnknown: false,
Query: query,
},
TranslationType: mode,
}
data, err := graphql.Post[searchData](ctx, c.httpClient, allAnimeBaseURL+"/api", searchQuery, vars, graphql.PostOptions{
Headers: map[string]string{
"Referer": allAnimeReferer,
"User-Agent": defaultUserAgent,
},
BodyMax: netutil.MiB2,
})
if err != nil {
return nil, err
}
out := make([]searchResult, 0, len(data.Shows.Edges))
for _, edge := range data.Shows.Edges {
id := edge.ID
malID := edge.MalID
name := edge.Name
if unquoted, err := strconv.Unquote("\"" + name + "\""); err == nil {
name = unquoted
}
name = strings.TrimSpace(name)
if id == "" {
continue
}
out = append(out, searchResult{ID: id, MalID: malID, Name: name})
}
return out, nil
}
func (c *AllAnimeProvider) resolveShowIDWithFallback(ctx context.Context, animeID int, titleCandidates []string, mode string) string {
targetMalIDStr := strconv.Itoa(animeID)
firstAvailableShowID := ""
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil || len(searchResults) == 0 {
continue
}
if showID := exactMatchShowID(searchResults, targetMalIDStr); showID != "" {
return showID
}
if firstAvailableShowID == "" {
firstAvailableShowID = searchResults[0].ID
}
}
return firstAvailableShowID
}
func exactMatchShowID(searchResults []searchResult, targetMalID string) string {
for _, res := range searchResults {
if res.MalID == targetMalID {
return res.ID
}
}
return ""
}
func (c *AllAnimeProvider) ResolveEpisodeProviderID(ctx context.Context, animeID int, titleCandidates []string) (string, error) {
for _, mode := range []string{"sub", "dub"} {
showID, err := c.resolveShowIDStrict(ctx, animeID, titleCandidates, mode)
if err == nil {
return showID, nil
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d", animeID)
}
func (c *AllAnimeProvider) resolveShowIDStrict(ctx context.Context, animeID int, titleCandidates []string, mode string) (string, error) {
targetMalIDStr := strconv.Itoa(animeID)
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
return res.ID, nil
}
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d in %s search", animeID, mode)
}

316
integrations/playback/allanime/sources.go Normal file
View File

@@ -0,0 +1,316 @@
package allanime
import (
"context"
"fmt"
"net/http"
"net/url"
"strings"
)
const episodeQueryHash = "d405d0edd690624b66baba3068e0edc3ac90f1597d898a1ec8db4e5c43c00fec"
type sourceReference struct {
URL string
Name string
}
func (c *AllAnimeProvider) GetEpisodeSources(ctx context.Context, showID string, episode string, mode string) ([]StreamSource, error) {
episodeQuery := `query($showId: String!, $translationType: VaildTranslationTypeEnumType!, $episodeString: String!) {
episode(showId: $showId, translationType: $translationType, episodeString: $episodeString) {
sourceUrls
}
}`
result, err := c.graphqlRequestWithHash(ctx, showID, episode, mode)
if err == nil {
sources := c.extractSourceURLsFromData(ctx, result)
if len(sources) > 0 {
return sources, nil
}
}
result, err = c.graphqlRequest(ctx, episodeQuery, map[string]any{
"showId": showID,
"translationType": mode,
"episodeString": episode,
})
if err != nil {
return nil, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid source response")
}
rawSourceURLs, ok := data["episode"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid episode response")
}
sourceURLs, ok := rawSourceURLs["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil, fmt.Errorf("no source urls")
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil, fmt.Errorf("no source references")
}
out := c.resolveSourceReferences(ctx, references)
if len(out) == 0 {
return nil, fmt.Errorf("no playable sources extracted")
}
return out, nil
}
func (c *AllAnimeProvider) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource {
episodeData, ok := data["episode"].(map[string]any)
if !ok {
return nil
}
sourceURLs, ok := episodeData["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil
}
return c.resolveSourceReferences(ctx, references)
}
func (c *AllAnimeProvider) resolveSourceReferences(ctx context.Context, references []sourceReference) []StreamSource {
out := make([]StreamSource, 0, len(references))
for _, ref := range references {
if source, ok := resolveDirectSource(ref); ok {
out = append(out, source)
return out
}
extracted := c.resolveExtractedSources(ctx, ref)
if len(extracted) > 0 {
out = append(out, extracted...)
return out
}
}
return out
}
func resolveDirectSource(ref sourceReference) (StreamSource, bool) {
target := strings.TrimSpace(ref.URL)
if target == "" {
return StreamSource{}, false
}
if isHTTPURL(target) {
if detectEmbedType(target) == "embed" {
return StreamSource{}, false
}
return buildStreamSource(target, detectSourceType(target), ref.Name), true
}
decoded := decodeSourceURL(target)
if !isHTTPURL(decoded) {
return StreamSource{}, false
}
if detectEmbedType(decoded) == "embed" {
return StreamSource{}, false
}
return buildStreamSource(decoded, detectSourceType(decoded), ref.Name), true
}
func (c *AllAnimeProvider) resolveExtractedSources(ctx context.Context, ref sourceReference) []StreamSource {
rawURL := strings.TrimSpace(ref.URL)
decoded := decodeSourceURL(rawURL)
if decoded == "" {
return nil
}
if isHTTPURL(decoded) {
extracted, err := c.extractor.ExtractEmbedVideoLinks(ctx, decoded)
if err != nil {
return nil
}
return extracted
}
if !strings.HasPrefix(decoded, "/") {
decoded = "/" + decoded
}
extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded)
if err != nil {
return nil
}
return extracted
}
func detectSourceType(sourceURL string) string {
sourceType := detectStreamType(sourceURL)
if sourceType != "unknown" {
return sourceType
}
return detectEmbedType(sourceURL)
}
func isHTTPURL(value string) bool {
return strings.HasPrefix(value, "http://") || strings.HasPrefix(value, "https://")
}
func buildStreamSource(url, sourceType, provider string) StreamSource {
return StreamSource{
URL: url,
Provider: provider,
Type: sourceType,
Referer: allAnimeReferer,
}
}
func buildSourceReferences(rawSourceURLs []any) []sourceReference {
priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"}
prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}}
prioritized := make(map[string]sourceReference)
fallback := make([]sourceReference, 0, len(rawSourceURLs))
seen := make(map[string]struct{})
for _, source := range rawSourceURLs {
item, ok := source.(map[string]any)
if !ok {
continue
}
sourceURL, _ := item["sourceUrl"].(string)
sourceName, _ := item["sourceName"].(string)
sourceURL = strings.TrimSpace(sourceURL)
sourceName = strings.TrimSpace(sourceName)
if sourceURL == "" {
continue
}
if _, exists := seen[sourceURL]; exists {
continue
}
seen[sourceURL] = struct{}{}
ref := sourceReference{URL: sourceURL, Name: sourceName}
normalized := strings.ToLower(sourceName)
if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider {
if _, exists := prioritized[normalized]; !exists {
prioritized[normalized] = ref
}
continue
}
fallback = append(fallback, ref)
}
ordered := make([]sourceReference, 0, len(prioritized)+len(fallback))
for _, provider := range priorityOrder {
if ref, ok := prioritized[provider]; ok {
ordered = append(ordered, ref)
}
}
ordered = append(ordered, fallback...)
return ordered
}
func (c *AllAnimeProvider) graphqlRequestWithHash(ctx context.Context, showID, episode, mode string) (map[string]any, error) {
req, err := newEpisodeHashRequest(ctx, showID, episode, mode)
if err != nil {
return nil, fmt.Errorf("create GET request: %w", err)
}
req.Header.Set("User-Agent", defaultUserAgent)
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Language", "en-US,en;q=0.5")
req.Header.Set("Accept-Encoding", "identity")
req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("Origin", allAnimeOrigin)
req.Header.Set("Sec-Fetch-Dest", "empty")
req.Header.Set("Sec-Fetch-Mode", "cors")
req.Header.Set("Sec-Fetch-Site", "cross-site")
statusCode, respBody, err := executeAndReadResponse(c.utlsClient, req, "execute GET request", "read response")
if err != nil {
return nil, err
}
if statusCode != http.StatusOK {
return nil, fmt.Errorf("GET status %d: %s", statusCode, string(respBody))
}
parsed, err := parseGraphQLResponse(respBody, "decode response")
if err != nil {
return nil, err
}
data, ok := parsed["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("no data in response")
}
decrypted, err := responseFromTobeparsed(data)
if err != nil {
return nil, err
}
if decrypted != nil {
return decrypted, nil
}
if hasEpisodeSourceURLs(data) {
return parsed, nil
}
return nil, fmt.Errorf("no usable data in response")
}
func newEpisodeHashRequest(ctx context.Context, showID, episode, mode string) (*http.Request, error) {
varsJSON := fmt.Sprintf(`{"showId":"%s","translationType":"%s","episodeString":"%s"}`, showID, strings.ToLower(mode), episode)
extJSON := fmt.Sprintf(`{"persistedQuery":{"version":1,"sha256Hash":"%s"}}`, episodeQueryHash)
params := url.Values{}
params.Set("variables", varsJSON)
params.Set("extensions", extJSON)
return http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s/api?%s", allAnimeBaseURL, params.Encode()), nil)
}
func detectStreamType(sourceURL string) string {
lower := strings.ToLower(sourceURL)
if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") {
return "m3u8"
}
if strings.Contains(lower, ".mp4") {
return "mp4"
}
return "unknown"
}
func detectEmbedType(rawURL string) string {
lower := strings.ToLower(rawURL)
embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"}
for _, host := range embedHosts {
if strings.Contains(lower, host) {
return "embed"
}
}
return "unknown"
}

391
internal/anime/browse_handler.go Normal file
View File

@@ -0,0 +1,391 @@
package anime
import (
"context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
)
type producerItem struct {
ID int `json:"id"`
Name string `json:"name"`
}
type browseQuery struct {
q string
animeType string
status string
orderBy string
sort string
sfw bool
studioID int
genres []int
page int
}
func producerQueryParams(c *gin.Context) (string, int, int, error) {
q := strings.TrimSpace(c.Query("q"))
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return "", 0, 0, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
limit, err := strconv.Atoi(c.DefaultQuery("limit", "50"))
if err != nil {
return "", 0, 0, fmt.Errorf("invalid limit")
}
if limit < 1 || limit > 12 {
limit = 12
}
return q, page, limit, nil
}
func producerItems(entries []jikan.ProducerListEntry) []producerItem {
items := make([]producerItem, 0, len(entries))
for _, producer := range entries {
name := jikan.ProducerListEntryName(producer)
if producer.MalID <= 0 || name == "" {
continue
}
items = append(items, producerItem{ID: producer.MalID, Name: name})
}
return items
}
func producerHTMLPayload(items []producerItem, hasNextPage bool, page int, q string, limit int) gin.H {
return gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": items,
"HasNextPage": hasNextPage,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
}
}
func requestWantsHTML(c *gin.Context) bool {
return strings.Contains(c.GetHeader("Accept"), "text/html")
}
func (h *AnimeHandler) HandleProducers(c *gin.Context) {
q, page, limit, err := producerQueryParams(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
res, err := h.svc.GetProducers(c.Request.Context(), q, page, limit)
if err != nil {
observability.WarnContext(c.Request.Context(),
"producers_fetch_failed",
"anime",
"",
map[string]any{
"q": q,
"page": page,
"limit": limit,
},
err,
)
if requestWantsHTML(c) {
c.HTML(http.StatusOK, "browse.gohtml", producerHTMLPayload([]producerItem{}, false, page, q, limit))
return
}
server.RespondError(
c,
http.StatusInternalServerError,
"producers_fetch_failed",
"anime",
"failed to load producers",
map[string]any{"q": q, "page": page, "limit": limit},
err,
)
return
}
items := producerItems(res.Items)
if requestWantsHTML(c) {
c.HTML(http.StatusOK, "browse.gohtml", producerHTMLPayload(items, res.HasNextPage, page, q, limit))
return
}
c.JSON(http.StatusOK, gin.H{
"items": items,
"hasNextPage": res.HasNextPage,
"nextPage": page + 1,
})
}
func parseBrowseQuery(c *gin.Context) (browseQuery, error) {
studioID := 0
if raw := strings.TrimSpace(c.Query("studio")); raw != "" {
id, err := strconv.Atoi(raw)
if err != nil || id < 0 {
return browseQuery{}, fmt.Errorf("invalid studio id")
}
studioID = id
}
genres := make([]int, 0, len(c.QueryArray("genres")))
for _, g := range c.QueryArray("genres") {
id, err := strconv.Atoi(g)
if err != nil {
return browseQuery{}, fmt.Errorf("invalid genre id")
}
if id > 0 {
genres = append(genres, id)
}
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return browseQuery{}, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
return browseQuery{
q: c.Query("q"),
animeType: c.Query("type"),
status: c.Query("status"),
orderBy: c.Query("order_by"),
sort: c.Query("sort"),
sfw: c.Query("sfw") != "false",
studioID: studioID,
genres: genres,
page: page,
}, nil
}
func browseStudioName(ctx context.Context, svc Service, studioID int) string {
if studioID <= 0 {
return ""
}
name, err := svc.GetProducerNameByID(ctx, studioID)
if err != nil {
return ""
}
return name
}
func browseTemplateData(
q browseQuery,
studioName string,
genresList []domain.Genre,
animes []domain.Anime,
user any,
watchlistMap map[int64]bool,
hasNextPage bool,
) gin.H {
return gin.H{
"CurrentPath": "/browse",
"Query": q.q,
"Type": q.animeType,
"Status": q.status,
"OrderBy": q.orderBy,
"Sort": q.sort,
"Genres": q.genres,
"Studio": q.studioID,
"StudioName": studioName,
"SFW": q.sfw,
"GenresList": genresList,
"Animes": animes,
"HasNextPage": hasNextPage,
"NextPage": q.page + 1,
"User": user,
"WatchlistMap": watchlistMap,
}
}
func (h *AnimeHandler) searchBrowse(ctx context.Context, query browseQuery) (jikan.SearchResult, error) {
return h.svc.SearchAdvanced(
ctx,
query.q,
query.animeType,
query.status,
query.orderBy,
query.sort,
query.genres,
query.studioID,
query.sfw,
query.page,
24,
)
}
func browseScrollData(
query browseQuery,
studioName string,
animes []domain.Anime,
watchlistMap map[int64]bool,
hasNextPage bool,
) gin.H {
return gin.H{
"_fragment": "anime_card_scroll",
"Animes": animes,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"Query": query.q,
"Type": query.animeType,
"Status": query.status,
"OrderBy": query.orderBy,
"Sort": query.sort,
"Genres": query.genres,
"Studio": query.studioID,
"StudioName": studioName,
"SFW": query.sfw,
"WatchlistMap": watchlistMap,
}
}
func (h *AnimeHandler) respondBrowseSearchError(c *gin.Context, query browseQuery, err error) {
server.RespondError(
c,
http.StatusInternalServerError,
"browse_search_failed",
"anime",
"failed to load browse results",
map[string]any{
"q": query.q,
"type": query.animeType,
"status": query.status,
"order_by": query.orderBy,
"sort": query.sort,
"studio": query.studioID,
"sfw": query.sfw,
"page": query.page,
},
err,
)
}
func (h *AnimeHandler) HandleBrowse(c *gin.Context) {
query, err := parseBrowseQuery(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
res, err := h.searchBrowse(c.Request.Context(), query)
if err != nil {
h.respondBrowseSearchError(c, query, err)
return
}
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
studioName := browseStudioName(c.Request.Context(), h.svc, query.studioID)
if c.GetHeader("HX-Request") == "true" && query.page > 1 {
c.HTML(http.StatusOK, "browse.gohtml", browseScrollData(query, studioName, animes, watchlistMap, res.HasNextPage))
return
}
genresList, _ := h.svc.GetGenres(c.Request.Context())
browseData := browseTemplateData(query, studioName, genresList, animes, user, watchlistMap, res.HasNextPage)
if c.GetHeader("HX-Request") == "true" {
browseData["_fragment"] = "browse_content"
c.HTML(http.StatusOK, "browse.gohtml", browseData)
return
}
c.HTML(http.StatusOK, "browse.gohtml", browseData)
}
type quickSearchResult struct {
ID int `json:"id"`
Title string `json:"title"`
Type string `json:"type"`
Year int `json:"year"`
Image string `json:"image"`
InWatchlist bool `json:"in_watchlist"`
}
func (h *AnimeHandler) HandleQuickSearch(c *gin.Context) {
query := c.Query("q")
if query == "" {
c.JSON(http.StatusOK, []any{})
return
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, 1, 5)
if err != nil {
c.JSON(http.StatusOK, []any{})
return
}
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
output := make([]quickSearchResult, len(animes))
for i, anime := range animes {
output[i] = quickSearchResult{
ID: anime.MalID,
Title: anime.DisplayTitle(),
Type: anime.Type,
Year: anime.Year,
Image: anime.ImageURL(),
InWatchlist: watchlistMap[int64(anime.MalID)],
}
}
c.JSON(http.StatusOK, output)
}
func (h *AnimeHandler) HandleRandomAnime(c *gin.Context) {
ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
defer cancel()
anime, err := h.svc.GetRandomAnime(ctx)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"random_anime_fetch_failed",
"anime",
"failed to fetch random anime",
nil,
err,
)
return
}
if anime.MalID == 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadGateway, "random anime unavailable")
return
}
inWatchlist := false
userID := server.CurrentUserID(c)
if userID != "" {
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, []int64{int64(anime.MalID)})
inWatchlist = watchlistMap[int64(anime.MalID)]
}
c.JSON(http.StatusOK, gin.H{
"data": anime,
"in_watchlist": inWatchlist,
})
}

123
internal/anime/catalog_handler.go Normal file
View File

@@ -0,0 +1,123 @@
package anime
import (
"mal/internal/observability"
"mal/internal/server"
"net/http"
"github.com/gin-gonic/gin"
)
func (h *AnimeHandler) HandleSearch(c *gin.Context) {
c.HTML(http.StatusOK, "search.gohtml", gin.H{
"User": server.CurrentUser(c),
"CurrentPath": "/search",
})
}
func (h *AnimeHandler) HandleCatalog(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "index.gohtml", gin.H{
"CurrentPath": "/",
"User": user,
"WatchlistMap": map[int64]bool{},
})
}
func (h *AnimeHandler) HandleCatalogAiring(c *gin.Context) {
h.renderCatalogSection(c, "Airing")
}
func (h *AnimeHandler) HandleCatalogPopular(c *gin.Context) {
h.renderCatalogSection(c, "Popular")
}
func (h *AnimeHandler) HandleCatalogContinue(c *gin.Context) {
h.renderCatalogSection(c, "Continue")
}
func (h *AnimeHandler) HandleCatalogTopPickForYou(c *gin.Context) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPickForYou(c.Request.Context(), userID)
if err != nil {
observability.WarnContext(c.Request.Context(),
"top_pick_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = "TopPickForYou"
data.Fragment = "top_pick_for_you_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) renderCatalogSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetCatalogSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "catalog_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "catalog_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) HandleTopPicks(c *gin.Context) {
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPicksForYou(c.Request.Context(), userID)
if err != nil {
observability.WarnContext(c.Request.Context(),
"top_picks_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
c.HTML(http.StatusOK, "top_picks.gohtml", gin.H{
"CurrentPath": "/top-picks",
"User": user,
"Animes": data.Animes,
"WatchlistMap": watchlistMap,
})
}
func (h *AnimeHandler) abortSectionFetch(c *gin.Context, event string, userID string, section string, err error) {
observability.WarnContext(c.Request.Context(),
event,
"anime",
"",
map[string]any{
"section": section,
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
}

69
internal/anime/command_palette.go
View File

@@ -1,19 +1,19 @@
package anime
import (
"context"
"fmt"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/server"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
)
const commandPaletteAnimeLimit = 24
type commandPaletteItem struct {
ID string `json:"id"`
Type string `json:"type"`
@@ -24,6 +24,12 @@ type commandPaletteItem struct {
Icon string `json:"icon,omitempty"`
}
type commandPaletteResponse struct {
Items []commandPaletteItem `json:"items"`
HasNextPage bool `json:"hasNextPage"`
NextPage int `json:"nextPage,omitempty"`
}
func (h *AnimeHandler) HandleCommandPalette(c *gin.Context) {
user := server.CurrentUser(c)
if user == nil {
@@ -32,41 +38,49 @@ func (h *AnimeHandler) HandleCommandPalette(c *gin.Context) {
}
query := strings.TrimSpace(c.Query("q"))
items := make([]commandPaletteItem, 0, 12)
if query != "" {
items = append(items, commandPaletteItem{
ID: "search:" + strings.ToLower(query),
Type: "search",
Label: fmt.Sprintf("Search anime for %q", query),
Subtitle: "Browse",
Href: "/browse?q=" + url.QueryEscape(query),
Icon: "search",
})
if len(query) >= 2 {
items = append(items, h.commandPaletteAnimeResults(c, query)...)
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil || page < 1 {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid page"})
return
}
items := make([]commandPaletteItem, 0, commandPaletteAnimeLimit)
if query != "" {
hasNextPage := false
if len(query) >= 2 {
var animeItems []commandPaletteItem
animeItems, hasNextPage = h.commandPaletteAnimeResults(c, query, page)
items = append(items, animeItems...)
}
if page == 1 {
items = append(items, h.commandPaletteNavigationItems(query)...)
items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...)
items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...)
c.JSON(http.StatusOK, items)
}
c.JSON(http.StatusOK, commandPaletteResponse{
Items: items,
HasNextPage: hasNextPage,
NextPage: page + 1,
})
return
}
items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...)
items = append(items, h.commandPaletteNavigationItems(query)...)
items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...)
c.JSON(http.StatusOK, items)
c.JSON(http.StatusOK, commandPaletteResponse{Items: items})
}
func (h *AnimeHandler) commandPaletteNavigationItems(query string) []commandPaletteItem {
all := []commandPaletteItem{
{ID: "nav:discover", Type: "navigation", Label: "Go to Discover", Subtitle: "Navigation", Href: "/discover", Icon: "compass"},
{ID: "nav:home", Type: "navigation", Label: "Go to Home", Subtitle: "Navigation", Href: "/", Icon: "home"},
{ID: "nav:watchlist", Type: "navigation", Label: "Go to Watchlist", Subtitle: "Navigation", Href: "/watchlist", Icon: "bookmark"},
{ID: "nav:popular", Type: "navigation", Label: "Browse popular", Subtitle: "Browse", Href: "/browse?order_by=popularity&sort=desc", Icon: "trending"},
{ID: "nav:airing", Type: "navigation", Label: "Currently airing", Subtitle: "Browse", Href: "/browse?status=airing&order_by=popularity&sort=desc", Icon: "play"},
{ID: "nav:top-picks", Type: "navigation", Label: "Open Top Picks", Subtitle: "Navigation", Href: "/top-picks", Icon: "sparkles"},
{ID: "nav:popular", Type: "navigation", Label: "Browse popular", Subtitle: "Browse", Href: "/browse?order_by=popularity&sort=asc", Icon: "trending"},
{ID: "nav:airing", Type: "navigation", Label: "Currently airing", Subtitle: "Browse", Href: "/browse?status=airing&order_by=popularity&sort=asc", Icon: "play"},
}
if query == "" {
return all
@@ -81,13 +95,10 @@ func (h *AnimeHandler) commandPaletteNavigationItems(query string) []commandPale
return filtered
}
func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string) []commandPaletteItem {
searchCtx, cancel := context.WithTimeout(c.Request.Context(), 800*time.Millisecond)
defer cancel()
res, err := h.svc.SearchAdvanced(searchCtx, query, "", "", "", "", nil, 0, true, 1, 5)
func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string, page int) ([]commandPaletteItem, bool) {
res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, page, commandPaletteAnimeLimit)
if err != nil {
return nil
return nil, false
}
animes := wrapAnimes(res.Animes)
@@ -102,7 +113,7 @@ func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string)
Image: anime.ImageURL(),
})
}
return items
return items, res.HasNextPage
}
func (h *AnimeHandler) commandPalettePersonalItems(c *gin.Context, userID string, query string) []commandPaletteItem {

220
internal/anime/details_handler.go Normal file
View File

@@ -0,0 +1,220 @@
package anime
import (
"context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"time"
"github.com/gin-gonic/gin"
)
const (
animeSectionTimeout = 12 * time.Second
watchOrderTimeout = 15 * time.Second
audioLookupTimeout = 8 * time.Second
)
func animeAudioAvailabilityLabel(episodes []domain.CanonicalEpisode) string {
hasKnownSub := false
for _, episode := range episodes {
if episode.HasDub {
return "Dub available"
}
if episode.HasSub || episode.SubOnly {
hasKnownSub = true
}
}
if hasKnownSub {
return "Subtitled only"
}
return ""
}
func (h *AnimeHandler) animeAudioAvailability(ctx context.Context, anime domain.Anime) string {
if h.episodeSvc == nil {
return ""
}
audioCtx, cancel := context.WithTimeout(ctx, audioLookupTimeout)
defer cancel()
episodeList, err := h.episodeSvc.GetCanonicalEpisodes(audioCtx, anime, true)
if err != nil {
observability.Warn(
"anime_audio_availability_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return ""
}
if episodeList.Source != "AllAnime" {
return ""
}
return animeAudioAvailabilityLabel(episodeList.Episodes)
}
func (h *AnimeHandler) HandleAnimeDetails(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
section := c.Query("section")
if section != "" && c.GetHeader("HX-Request") == "true" {
h.handleAnimeDetailsSection(c, id, section)
return
}
anime, err := h.svc.GetAnimeByID(c.Request.Context(), id)
if err != nil {
c.Status(http.StatusNotFound)
return
}
h.svc.WarmDetailSections(id)
user := server.CurrentUser(c)
status := ""
var watchlistIDs []int64
ep := 0
var cwSeconds float64
if user != nil {
entry, err := h.watchlistSvc.GetWatchListEntry(c.Request.Context(), user.ID, int64(id))
if err == nil {
status = entry.Status
watchlistIDs = []int64{entry.AnimeID}
}
cwEntry, err := h.watchlistSvc.GetContinueWatchingEntry(c.Request.Context(), user.ID, int64(id))
if err == nil && cwEntry.CurrentEpisode.Valid {
ep = int(cwEntry.CurrentEpisode.Int64)
cwSeconds = cwEntry.CurrentTimeSeconds
}
}
audioAvailability := h.animeAudioAvailability(c.Request.Context(), anime)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"Anime": anime,
"AudioAvailability": audioAvailability,
"CurrentPath": fmt.Sprintf("/anime/%d", id),
"User": user,
"Status": status,
"WatchlistIDs": watchlistIDs,
"ContinueWatchingEp": ep,
"ContinueWatchingTime": cwSeconds,
})
}
func (h *AnimeHandler) handleAnimeDetailsSection(c *gin.Context, id int, section string) {
sectionCtx, cancel := context.WithTimeout(c.Request.Context(), animeSectionTimeout)
defer cancel()
data, tplName, err := h.loadAnimeDetailsSection(sectionCtx, id, section)
if err != nil {
observability.Warn(
"anime_section_fetch_failed",
"anime",
"",
map[string]any{
"section": section,
"anime_id": id,
},
err,
)
if section == "recommendations" {
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "anime_recommendations_loading",
"AnimeID": id,
})
return
}
c.Status(http.StatusNoContent)
return
}
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": tplName,
"Items": data,
})
}
func (h *AnimeHandler) loadAnimeDetailsSection(ctx context.Context, id int, section string) (any, string, error) {
switch section {
case "characters":
data, err := h.svc.GetCharacters(ctx, id)
return data, "anime_characters", err
case "recommendations":
data, err := h.svc.GetRecommendations(ctx, id)
return data, "anime_recommendations", err
case "statistics":
data, err := h.svc.GetStatistics(ctx, id)
return data, "anime_statistics", err
case "themes":
data, err := h.svc.GetThemes(ctx, id)
return data, "anime_themes", err
default:
return nil, "", nil
}
}
func (h *AnimeHandler) HandleHTMLWatchOrder(c *gin.Context) {
id, err := strconv.Atoi(c.Query("animeId"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
userID := server.CurrentUserID(c)
mode := jikan.NormalizeWatchOrderMode(c.Query("mode"))
relationsCtx, cancel := context.WithTimeout(c.Request.Context(), watchOrderTimeout)
defer cancel()
relations, err := h.svc.GetRelations(relationsCtx, id, mode)
if err != nil {
observability.Warn(
"relations_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": id,
},
err,
)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order_loading",
"AnimeID": id,
"Mode": string(mode),
})
return
}
relationAnimeIDs := make([]int64, 0, len(relations))
for _, relation := range relations {
if relation.Anime.MalID > 0 {
relationAnimeIDs = append(relationAnimeIDs, int64(relation.Anime.MalID))
}
}
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, relationAnimeIDs)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order",
"Relations": relations,
"AnimeID": id,
"Mode": string(mode),
"WatchlistMap": watchlistMap,
})
}

770
internal/anime/handler.go
View File

@@ -2,38 +2,22 @@ package anime
import (
"context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
)
const (
animeSectionTimeout = 12 * time.Second
watchOrderTimeout = 15 * time.Second
audioLookupTimeout = 8 * time.Second
)
type AnimeHandler struct {
svc Service
watchlistSvc domain.WatchlistService
episodeSvc domain.EpisodeService
scheduleCacheMu sync.Mutex
scheduleCache map[string]cachedWeekSchedule
sync.Mutex
}
type Service interface {
domain.AnimeCatalogService
domain.AnimeDiscoverService
domain.AnimeSearchService
domain.AnimeDetailsService
WarmDetailSections(id int)
@@ -44,7 +28,7 @@ func NewAnimeHandler(svc Service, watchlistSvc domain.WatchlistService, episodeS
svc: svc,
watchlistSvc: watchlistSvc,
episodeSvc: episodeSvc,
scheduleCache: map[string]cachedWeekSchedule{},
scheduleCache: make(map[string]cachedWeekSchedule),
}
}
@@ -70,63 +54,14 @@ func (h *AnimeHandler) watchlistMapForIDs(ctx context.Context, userID string, an
return watchlistMap
}
func animeAudioAvailabilityLabel(episodes []domain.CanonicalEpisode) string {
hasKnownSub := false
for _, episode := range episodes {
if episode.HasDub {
return "Dub available"
}
if episode.HasSub || episode.SubOnly {
hasKnownSub = true
}
}
if hasKnownSub {
return "Subtitled only"
}
return ""
}
func (h *AnimeHandler) animeAudioAvailability(ctx context.Context, anime domain.Anime) string {
if h.episodeSvc == nil {
return ""
}
audioCtx, cancel := context.WithTimeout(ctx, audioLookupTimeout)
defer cancel()
episodeList, err := h.episodeSvc.GetCanonicalEpisodes(audioCtx, anime, true)
if err != nil {
observability.Warn(
"anime_audio_availability_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return ""
}
if episodeList.Source != "AllAnime" {
return ""
}
return animeAudioAvailabilityLabel(episodeList.Episodes)
}
func (h *AnimeHandler) Register(r *gin.Engine) {
r.GET("/", h.HandleCatalog)
r.GET("/api/catalog/airing", h.HandleCatalogAiring)
r.GET("/api/catalog/popular", h.HandleCatalogPopular)
r.GET("/api/catalog/continue", h.HandleCatalogContinue)
r.GET("/api/catalog/top-pick", h.HandleCatalogTopPickForYou)
r.GET("/discover", h.HandleDiscover)
r.GET("/discover/top-picks", h.HandleDiscoverTopPicksForYou)
r.GET("/api/discover/trending", h.HandleDiscoverTrending)
r.GET("/api/discover/upcoming", h.HandleDiscoverUpcoming)
r.GET("/api/discover/top", h.HandleDiscoverTop)
r.GET("/schedule", h.HandleSchedule)
r.GET("/api/schedule", h.HandleScheduleSection)
r.GET("/search", h.HandleSearch)
r.GET("/top-picks", h.HandleTopPicks)
r.GET("/browse", h.HandleBrowse)
r.GET("/anime/:id", h.HandleAnimeDetails)
r.GET("/anime/:id/reviews", h.HandleAnimeReviews)
@@ -136,700 +71,3 @@ func (h *AnimeHandler) Register(r *gin.Engine) {
r.GET("/api/jikan/random/anime", h.HandleRandomAnime)
r.GET("/api/jikan/producers", h.HandleProducers)
}
func (h *AnimeHandler) HandleProducers(c *gin.Context) {
q := strings.TrimSpace(c.Query("q"))
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
limit, err := strconv.Atoi(c.DefaultQuery("limit", "50"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid limit")
return
}
if limit < 1 {
limit = 12
}
if limit > 12 {
limit = 12
}
res, err := h.svc.GetProducers(c.Request.Context(), q, page, limit)
if err != nil {
observability.Warn(
"producers_fetch_failed",
"anime",
"",
map[string]any{
"q": q,
"page": page,
"limit": limit,
},
err,
)
if strings.Contains(c.GetHeader("Accept"), "text/html") {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": []any{},
"HasNextPage": false,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
})
return
}
server.RespondError(
c,
http.StatusInternalServerError,
"producers_fetch_failed",
"anime",
"failed to load producers",
map[string]any{"q": q, "page": page, "limit": limit},
err,
)
return
}
type item struct {
ID int `json:"id"`
Name string `json:"name"`
}
items := make([]item, 0, len(res.Items))
for _, p := range res.Items {
name := jikan.ProducerListEntryName(p)
if p.MalID <= 0 || name == "" {
continue
}
items = append(items, item{ID: p.MalID, Name: name})
}
if strings.Contains(c.GetHeader("Accept"), "text/html") {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": items,
"HasNextPage": res.HasNextPage,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
})
return
}
c.JSON(http.StatusOK, gin.H{
"items": items,
"hasNextPage": res.HasNextPage,
"nextPage": page + 1,
})
}
func (h *AnimeHandler) HandleCatalog(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "index.gohtml", gin.H{
"CurrentPath": "/",
"User": user,
"WatchlistMap": map[int64]bool{},
})
}
func (h *AnimeHandler) HandleCatalogAiring(c *gin.Context) {
h.renderCatalogSection(c, "Airing")
}
func (h *AnimeHandler) HandleCatalogPopular(c *gin.Context) {
h.renderCatalogSection(c, "Popular")
}
func (h *AnimeHandler) HandleCatalogContinue(c *gin.Context) {
h.renderCatalogSection(c, "Continue")
}
func (h *AnimeHandler) HandleCatalogTopPickForYou(c *gin.Context) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPickForYou(c.Request.Context(), userID)
if err != nil {
observability.Warn(
"top_pick_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = "TopPickForYou"
data.Fragment = "top_pick_for_you_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) renderCatalogSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetCatalogSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "catalog_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "catalog_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) HandleDiscover(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "discover.gohtml", gin.H{
"CurrentPath": "/discover",
"User": user,
})
}
func (h *AnimeHandler) HandleDiscoverTopPicksForYou(c *gin.Context) {
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPicksForYou(c.Request.Context(), userID)
if err != nil {
observability.Warn(
"top_picks_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
c.HTML(http.StatusOK, "discover.gohtml", gin.H{
"_fragment": "",
"CurrentPath": "/discover",
"User": user,
"Animes": data.Animes,
"WatchlistMap": watchlistMap,
"IsTopPicks": true,
})
}
func (h *AnimeHandler) HandleDiscoverTrending(c *gin.Context) {
h.renderDiscoverSection(c, "Trending")
}
func (h *AnimeHandler) HandleDiscoverUpcoming(c *gin.Context) {
h.renderDiscoverSection(c, "Upcoming")
}
func (h *AnimeHandler) HandleDiscoverTop(c *gin.Context) {
h.renderDiscoverSection(c, "Top")
}
func (h *AnimeHandler) renderDiscoverSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetDiscoverSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "discover_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "discover_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "discover.gohtml", data)
}
func (h *AnimeHandler) abortSectionFetch(c *gin.Context, event string, userID string, section string, err error) {
observability.Warn(
event,
"anime",
"",
map[string]any{
"section": section,
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
}
func (h *AnimeHandler) HandleSchedule(c *gin.Context) {
user := server.CurrentUser(c)
year, week := parseYearWeek(c)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"CurrentPath": "/schedule",
"User": user,
"ScheduleYear": year,
"ScheduleWeek": week,
})
}
func (h *AnimeHandler) HandleScheduleSection(c *gin.Context) {
year, week := parseYearWeek(c)
timezone := scheduleTimezone(c)
schedule, err := h.getCachedAnimeScheduleWeek(c.Request.Context(), year, week, timezone)
if err != nil {
prevYear, prevWeek := adjacentISOWeek(year, week, -1)
nextYear, nextWeek := adjacentISOWeek(year, week, 1)
observability.Warn(
"animeschedule_fetch_failed",
"anime",
"",
map[string]any{
"year": year,
"week": week,
"timezone": timezone,
},
err,
)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"_fragment": "schedule_section_scraped",
"ScheduleDays": []any{},
"ScheduleYear": year,
"ScheduleWeek": week,
"PrevYear": prevYear,
"PrevWeek": prevWeek,
"NextYear": nextYear,
"NextWeek": nextWeek,
"ScheduleError": true,
})
return
}
days := buildScheduleDays(schedule, schedule.Year, schedule.Week)
prevYear, prevWeek := adjacentISOWeek(schedule.Year, schedule.Week, -1)
nextYear, nextWeek := adjacentISOWeek(schedule.Year, schedule.Week, 1)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"_fragment": "schedule_section_scraped",
"ScheduleDays": days,
"ScheduleYear": schedule.Year,
"ScheduleWeek": schedule.Week,
"PrevYear": prevYear,
"PrevWeek": prevWeek,
"NextYear": nextYear,
"NextWeek": nextWeek,
})
}
func (h *AnimeHandler) HandleBrowse(c *gin.Context) {
q := c.Query("q")
animeType := c.Query("type")
status := c.Query("status")
orderBy := c.Query("order_by")
sort := c.Query("sort")
sfw := c.Query("sfw") != "false"
studioID := 0
if raw := strings.TrimSpace(c.Query("studio")); raw != "" {
id, err := strconv.Atoi(raw)
if err != nil || id < 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid studio id")
return
}
studioID = id
}
var genres []int
for _, g := range c.QueryArray("genres") {
id, err := strconv.Atoi(g)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid genre id")
return
}
if id > 0 {
genres = append(genres, id)
}
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), q, animeType, status, orderBy, sort, genres, studioID, sfw, page, 24)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"browse_search_failed",
"anime",
"failed to load browse results",
map[string]any{
"q": q,
"type": animeType,
"status": status,
"order_by": orderBy,
"sort": sort,
"studio": studioID,
"sfw": sfw,
"page": page,
},
err,
)
return
}
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
studioName := ""
if studioID > 0 {
name, err := h.svc.GetProducerNameByID(c.Request.Context(), studioID)
if err == nil {
studioName = name
}
}
if c.GetHeader("HX-Request") == "true" && page > 1 {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "anime_card_scroll",
"Animes": animes,
"NextPage": page + 1,
"HasNextPage": res.HasNextPage,
"Query": q,
"Type": animeType,
"Status": status,
"OrderBy": orderBy,
"Sort": sort,
"Genres": genres,
"Studio": studioID,
"StudioName": studioName,
"SFW": sfw,
"WatchlistMap": watchlistMap,
})
return
}
genresList, _ := h.svc.GetGenres(c.Request.Context())
browseData := gin.H{
"CurrentPath": "/browse",
"Query": q,
"Type": animeType,
"Status": status,
"OrderBy": orderBy,
"Sort": sort,
"Genres": genres,
"Studio": studioID,
"StudioName": studioName,
"SFW": sfw,
"GenresList": genresList,
"Animes": animes,
"HasNextPage": res.HasNextPage,
"NextPage": page + 1,
"User": user,
"WatchlistMap": watchlistMap,
}
if c.GetHeader("HX-Request") == "true" {
browseData["_fragment"] = "browse_content"
c.HTML(http.StatusOK, "browse.gohtml", browseData)
return
}
c.HTML(http.StatusOK, "browse.gohtml", browseData)
}
func (h *AnimeHandler) HandleAnimeDetails(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
section := c.Query("section")
if section != "" && c.GetHeader("HX-Request") == "true" {
sectionCtx, cancel := context.WithTimeout(c.Request.Context(), animeSectionTimeout)
defer cancel()
var data any
var tplName string
var err error
switch section {
case "characters":
data, err = h.svc.GetCharacters(sectionCtx, id)
tplName = "anime_characters"
case "recommendations":
data, err = h.svc.GetRecommendations(sectionCtx, id)
tplName = "anime_recommendations"
case "statistics":
data, err = h.svc.GetStatistics(sectionCtx, id)
tplName = "anime_statistics"
case "themes":
data, err = h.svc.GetThemes(sectionCtx, id)
tplName = "anime_themes"
}
if err != nil {
observability.Warn(
"anime_section_fetch_failed",
"anime",
"",
map[string]any{
"section": section,
"anime_id": id,
},
err,
)
if section == "recommendations" {
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "anime_recommendations_loading",
"AnimeID": id,
})
return
}
c.Status(http.StatusNoContent)
return
}
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": tplName,
"Items": data,
})
return
}
anime, err := h.svc.GetAnimeByID(c.Request.Context(), id)
if err != nil {
c.Status(http.StatusNotFound)
return
}
h.svc.WarmDetailSections(id)
user := server.CurrentUser(c)
status := ""
var watchlistIDs []int64
ep := 0
var cwSeconds float64
if user != nil {
entry, err := h.watchlistSvc.GetWatchListEntry(c.Request.Context(), user.ID, int64(id))
if err == nil {
status = entry.Status
watchlistIDs = []int64{entry.AnimeID}
}
cwEntry, err := h.watchlistSvc.GetContinueWatchingEntry(c.Request.Context(), user.ID, int64(id))
if err == nil && cwEntry.CurrentEpisode.Valid {
ep = int(cwEntry.CurrentEpisode.Int64)
cwSeconds = cwEntry.CurrentTimeSeconds
}
}
audioAvailability := h.animeAudioAvailability(c.Request.Context(), anime)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"Anime": anime,
"AudioAvailability": audioAvailability,
"CurrentPath": fmt.Sprintf("/anime/%d", id),
"User": user,
"Status": status,
"WatchlistIDs": watchlistIDs,
"ContinueWatchingEp": ep,
"ContinueWatchingTime": cwSeconds,
})
}
func (h *AnimeHandler) HandleHTMLWatchOrder(c *gin.Context) {
id, err := strconv.Atoi(c.Query("animeId"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
userID := server.CurrentUserID(c)
relationsCtx, cancel := context.WithTimeout(c.Request.Context(), watchOrderTimeout)
defer cancel()
relations, err := h.svc.GetRelations(relationsCtx, id)
if err != nil {
observability.Warn(
"relations_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": id,
},
err,
)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order_loading",
"AnimeID": id,
})
return
}
relationAnimeIDs := make([]int64, 0, len(relations))
for _, relation := range relations {
if relation.Anime.MalID > 0 {
relationAnimeIDs = append(relationAnimeIDs, int64(relation.Anime.MalID))
}
}
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, relationAnimeIDs)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order",
"Relations": relations,
"AnimeID": id,
"WatchlistMap": watchlistMap,
})
}
func (h *AnimeHandler) HandleQuickSearch(c *gin.Context) {
query := c.Query("q")
if query == "" {
c.JSON(http.StatusOK, []any{})
return
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, 1, 5)
if err != nil {
c.JSON(http.StatusOK, []any{})
return
}
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
type quickSearchResult struct {
ID int `json:"id"`
Title string `json:"title"`
Type string `json:"type"`
Year int `json:"year"`
Image string `json:"image"`
InWatchlist bool `json:"in_watchlist"`
}
output := make([]quickSearchResult, len(animes))
for i, anime := range animes {
output[i] = quickSearchResult{
ID: anime.MalID,
Title: anime.DisplayTitle(),
Type: anime.Type,
Year: anime.Year,
Image: anime.ImageURL(),
InWatchlist: watchlistMap[int64(anime.MalID)],
}
}
c.JSON(http.StatusOK, output)
}
func (h *AnimeHandler) HandleRandomAnime(c *gin.Context) {
ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
defer cancel()
anime, err := h.svc.GetRandomAnime(ctx)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"random_anime_fetch_failed",
"anime",
"failed to fetch random anime",
nil,
err,
)
return
}
if anime.MalID == 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadGateway, "random anime unavailable")
return
}
inWatchlist := false
userID := server.CurrentUserID(c)
if userID != "" {
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, []int64{int64(anime.MalID)})
inWatchlist = watchlistMap[int64(anime.MalID)]
}
c.JSON(http.StatusOK, gin.H{
"data": anime,
"in_watchlist": inWatchlist,
})
}
func (h *AnimeHandler) HandleAnimeReviews(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
reviews, hasNextPage, err := h.svc.GetReviews(c.Request.Context(), id, page)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"anime_reviews_fetch_failed",
"anime",
"failed to load reviews",
map[string]any{"anime_id": id, "page": page},
err,
)
return
}
user := server.CurrentUser(c)
if c.GetHeader("HX-Request") == "true" && page > 1 {
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"_fragment": "review_cards",
"Reviews": reviews,
"NextPage": page + 1,
"HasNextPage": hasNextPage,
"AnimeID": id,
})
return
}
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"CurrentPath": fmt.Sprintf("/anime/%d/reviews", id),
"Reviews": reviews,
"NextPage": page + 1,
"HasNextPage": hasNextPage,
"AnimeID": id,
"User": user,
})
}

1
internal/anime/module.go
View File

@@ -14,7 +14,6 @@ var Module = fx.Options(
NewAnimeService,
fx.As(new(Service)),
fx.As(new(domain.AnimeCatalogService)),
fx.As(new(domain.AnimeDiscoverService)),
fx.As(new(domain.AnimeSearchService)),
fx.As(new(domain.AnimeDetailsService)),
fx.As(new(domain.AnimePlaybackService)),

354
internal/anime/recommendations.go
View File

@@ -1,14 +1,19 @@
package anime
import (
"context"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
"math"
"slices"
"sort"
"strings"
"sync"
"time"
"golang.org/x/sync/errgroup"
)
const (
@@ -270,6 +275,24 @@ func scoreRecommendationCandidate(
score += themeScore * forYouThemeMatchWeight
score += studioScore * forYouStudioMatchWeight
score += demographicScore * forYouDemographicMatchWeight
score += recommendationCandidateScoreAdjustments(now, profile, candidate)
return recommendationCandidate{
anime: candidate,
score: score,
genreMatches: genreMatches,
themeMatches: themeMatches,
studioMatches: studioMatches,
demographicMatches: demographicMatches,
}
}
func recommendationCandidateScoreAdjustments(
now time.Time,
profile userTasteProfile,
candidate jikan.Anime,
) float64 {
var score float64
if candidate.Score > 0 {
score += min(candidate.Score/10.0, 1.0)
@@ -280,31 +303,41 @@ func scoreRecommendationCandidate(
if profile.prefersAiring && candidate.Airing {
score += 0.5
}
if profile.prefersRecent && candidate.Year > 0 && now.Year()-candidate.Year <= 4 {
if profile.prefersRecent && isRecentCandidate(now, candidate.Year) {
score += 0.45
}
if candidate.Year > 0 && now.Year()-candidate.Year > 15 {
if isClassicCandidate(now, candidate.Year) {
score -= 0.2
}
if candidate.Status == "Not yet aired" {
score -= 0.35
}
if candidate.Aired.From != "" {
if airedAt, err := time.Parse(time.RFC3339, candidate.Aired.From); err == nil {
if now.Sub(airedAt) <= forYouFreshReleaseWindow {
if isFreshRelease(now, candidate.Aired.From) {
score += 0.3
}
}
return score
}
return recommendationCandidate{
anime: candidate,
score: score,
genreMatches: genreMatches,
themeMatches: themeMatches,
studioMatches: studioMatches,
demographicMatches: demographicMatches,
func isRecentCandidate(now time.Time, year int) bool {
return year > 0 && now.Year()-year <= 4
}
func isClassicCandidate(now time.Time, year int) bool {
return year > 0 && now.Year()-year > 15
}
func isFreshRelease(now time.Time, airedFrom string) bool {
if airedFrom == "" {
return false
}
airedAt, err := time.Parse(time.RFC3339, airedFrom)
if err != nil {
return false
}
return now.Sub(airedAt) <= forYouFreshReleaseWindow
}
func weightedEntityMatch(weights map[int]float64, entities []jikan.NamedEntity) (int, float64) {
@@ -501,3 +534,298 @@ func recentFeatureCounts(
}
return counts
}
type rankedCandidate struct {
id int
collaborativeScore float64
profileSearchScore float64
anime jikan.Anime
hasAnime bool
}
type candidateStore struct {
watchlistAnimeIDs map[int]struct{}
byID map[int]rankedCandidate
mu sync.Mutex
}
func newCandidateStore(watchlist []db.GetUserWatchListRow) *candidateStore {
watchlistAnimeIDs := make(map[int]struct{}, len(watchlist))
for _, entry := range watchlist {
if entry.AnimeID <= 0 {
continue
}
watchlistAnimeIDs[int(entry.AnimeID)] = struct{}{}
}
return &candidateStore{
watchlistAnimeIDs: watchlistAnimeIDs,
byID: map[int]rankedCandidate{},
}
}
func (s *candidateStore) upsert(candidate rankedCandidate) {
if candidate.id <= 0 {
return
}
if _, exists := s.watchlistAnimeIDs[candidate.id]; exists {
return
}
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.byID[candidate.id]
if !ok {
s.byID[candidate.id] = candidate
return
}
current.collaborativeScore += candidate.collaborativeScore
current.profileSearchScore += candidate.profileSearchScore
if candidate.hasAnime {
current.anime = candidate.anime
current.hasAnime = true
}
s.byID[candidate.id] = current
}
func (s *candidateStore) ranked() []rankedCandidate {
ranked := make([]rankedCandidate, 0, len(s.byID))
for _, item := range s.byID {
ranked = append(ranked, item)
}
sort.Slice(ranked, func(i, j int) bool {
left := rankedCandidateRetrievalScore(ranked[i].collaborativeScore, ranked[i].profileSearchScore)
right := rankedCandidateRetrievalScore(ranked[j].collaborativeScore, ranked[j].profileSearchScore)
if left == right {
return ranked[i].id < ranked[j].id
}
return left > right
})
return ranked
}
func (s *animeService) GetTopPickForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouResultLimit)
}
func (s *animeService) GetTopPicksForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouFullResultLimit)
}
func (s *animeService) fetchSeedAnimes(ctx context.Context, seedPool []recommendationSeed) ([]jikan.Anime, error) {
seedAnimes := make([]jikan.Anime, len(seedPool))
var g errgroup.Group
g.SetLimit(4)
for i, seed := range seedPool {
g.Go(func() error {
anime, err := s.jikan.GetAnimeByID(ctx, seed.animeID)
if err != nil {
return err
}
seedAnimes[i] = anime
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return seedAnimes, nil
}
func (s *animeService) collectCollaborativeCandidates(ctx context.Context, seedPool []recommendationSeed, store *candidateStore) error {
var g errgroup.Group
g.SetLimit(4)
for _, seed := range seedPool {
g.Go(func() error {
recs, err := s.jikan.GetAnimeRecommendations(ctx, seed.animeID)
if err != nil {
return err
}
for i, rec := range recs {
if i >= forYouMaxRecommendations {
break
}
id := rec.Entry.MalID
if id <= 0 || id == seed.animeID {
continue
}
store.upsert(rankedCandidate{
id: id,
collaborativeScore: float64(rec.Votes) * seed.weight,
})
}
return nil
})
}
return g.Wait()
}
func (s *animeService) collectProfileSearchCandidates(ctx context.Context, profile userTasteProfile, store *candidateStore) error {
queries := buildProfileSearchQueries(profile)
var g errgroup.Group
g.SetLimit(3)
for _, query := range queries {
g.Go(func() error {
res, err := s.jikan.SearchAdvanced(
ctx,
"",
"",
"",
"score",
"desc",
query.genreIDs,
query.studioID,
true,
1,
forYouProfileSearchLimit,
)
if err != nil {
observability.Warn(
"top_pick_profile_search_failed",
"anime",
"",
map[string]any{
"genres": query.genreIDs,
"studio_id": query.studioID,
},
err,
)
return nil
}
for i, anime := range res.Animes {
if anime.MalID <= 0 {
continue
}
store.upsert(rankedCandidate{
id: anime.MalID,
profileSearchScore: query.weight * profileSearchRankWeight(i),
anime: anime,
hasAnime: true,
})
}
return nil
})
}
return g.Wait()
}
func (s *animeService) scoreRankedCandidates(
ctx context.Context,
now time.Time,
profile userTasteProfile,
ranked []rankedCandidate,
) ([]recommendationCandidate, error) {
limit := min(len(ranked), forYouCandidateFetchLimit)
candidates := make([]recommendationCandidate, 0, limit)
var candidatesMu sync.Mutex
var g errgroup.Group
g.SetLimit(6)
for i := 0; i < limit; i++ {
item := ranked[i]
g.Go(func() error {
anime := item.anime
if !item.hasAnime || !hasTasteMetadata(anime) {
fetchedAnime, err := s.jikan.GetAnimeByID(ctx, item.id)
if err != nil {
observability.Warn(
"recommendation_anime_fetch_failed",
"anime",
"",
map[string]any{"anime_id": item.id},
err,
)
return nil
}
anime = fetchedAnime
}
candidate := scoreRecommendationCandidate(
now,
profile,
anime,
item.collaborativeScore,
item.profileSearchScore,
)
candidatesMu.Lock()
candidates = append(candidates, candidate)
candidatesMu.Unlock()
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].score == candidates[j].score {
return candidates[i].anime.MalID < candidates[j].anime.MalID
}
return candidates[i].score > candidates[j].score
})
return candidates, nil
}
func (s *animeService) getTopPicksForYou(
ctx context.Context,
userID string,
resultLimit int,
) (domain.CatalogSectionData, error) {
if strings.TrimSpace(userID) == "" {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return domain.CatalogSectionData{}, err
}
now := time.Now()
seedPool := buildRecommendationSeeds(now, watchlist)
if len(seedPool) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
seedAnimes, err := s.fetchSeedAnimes(ctx, seedPool)
if err != nil {
return domain.CatalogSectionData{}, err
}
profile := buildTasteProfile(now, seedPool, seedAnimes)
store := newCandidateStore(watchlist)
if err := s.collectCollaborativeCandidates(ctx, seedPool, store); err != nil {
return domain.CatalogSectionData{}, err
}
if err := s.collectProfileSearchCandidates(ctx, profile, store); err != nil {
return domain.CatalogSectionData{}, err
}
ranked := store.ranked()
if len(ranked) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
candidates, err := s.scoreRankedCandidates(ctx, now, profile, ranked)
if err != nil {
return domain.CatalogSectionData{}, err
}
return domain.CatalogSectionData{
Animes: rerankRecommendationCandidates(candidates, resultLimit),
}, nil
}

76
internal/anime/reviews_handler.go Normal file
View File

@@ -0,0 +1,76 @@
package anime
import (
"fmt"
"mal/internal/server"
"net/http"
"strconv"
"github.com/gin-gonic/gin"
)
type reviewsQuery struct {
animeID int
page int
}
func parseReviewsQuery(c *gin.Context) (reviewsQuery, error) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
return reviewsQuery{}, fmt.Errorf("invalid anime id")
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return reviewsQuery{}, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
return reviewsQuery{animeID: id, page: page}, nil
}
func (h *AnimeHandler) HandleAnimeReviews(c *gin.Context) {
query, err := parseReviewsQuery(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
reviews, hasNextPage, err := h.svc.GetReviews(c.Request.Context(), query.animeID, query.page)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"anime_reviews_fetch_failed",
"anime",
"failed to load reviews",
map[string]any{"anime_id": query.animeID, "page": query.page},
err,
)
return
}
user := server.CurrentUser(c)
if c.GetHeader("HX-Request") == "true" && query.page > 1 {
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"_fragment": "review_cards",
"Reviews": reviews,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"AnimeID": query.animeID,
})
return
}
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"CurrentPath": fmt.Sprintf("/anime/%d/reviews", query.animeID),
"Reviews": reviews,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"AnimeID": query.animeID,
"User": user,
})
}

8
internal/anime/schedule.go
View File

@@ -45,9 +45,9 @@ func (h *AnimeHandler) getCachedAnimeScheduleWeek(ctx context.Context, year int,
cacheKey := fmt.Sprintf("%d-%02d-%s", year, week, timezone)
const ttl = 10 * time.Minute
h.scheduleCacheMu.Lock()
h.Lock()
cached, ok := h.scheduleCache[cacheKey]
h.scheduleCacheMu.Unlock()
h.Unlock()
if ok && time.Since(cached.fetchedAt) < ttl {
return cached.value, nil
@@ -58,9 +58,9 @@ func (h *AnimeHandler) getCachedAnimeScheduleWeek(ctx context.Context, year int,
return animeschedule.WeekSchedule{}, err
}
h.scheduleCacheMu.Lock()
h.Lock()
h.scheduleCache[cacheKey] = cachedWeekSchedule{fetchedAt: time.Now(), value: value}
h.scheduleCacheMu.Unlock()
h.Unlock()
return value, nil
}

359
internal/anime/service.go
View File

@@ -1,17 +1,13 @@
// Package anime provides anime catalog, discovery, search, and details services.
// Package anime provides anime catalog, search, and details services.
package anime
import (
"context"
"errors"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
"math/rand"
"sort"
"strings"
"sync"
"time"
"golang.org/x/sync/errgroup"
@@ -76,355 +72,6 @@ func (s *animeService) GetCatalogSection(ctx context.Context, userID string, sec
}, nil
}
func (s *animeService) GetDiscoverSection(ctx context.Context, userID string, section string) (domain.DiscoverSectionData, error) {
var res jikan.TopAnimeResult
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
switch section {
case "Trending":
res, err = s.jikan.GetSeasonsNow(gCtx, 1)
case "Upcoming":
res, err = s.jikan.GetSeasonsUpcoming(gCtx, 1)
case "Top":
res, err = s.jikan.GetTopAnime(gCtx, 1)
}
return err
})
if err := g.Wait(); err != nil {
return domain.DiscoverSectionData{}, err
}
animes := wrapAnimes(res.Animes)
if len(animes) > 8 {
animes = animes[:8]
}
return domain.DiscoverSectionData{
Animes: animes,
}, nil
}
func (s *animeService) GetTopPickForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouResultLimit)
}
func (s *animeService) GetTopPicksForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouFullResultLimit)
}
func (s *animeService) getTopPicksForYou(
ctx context.Context,
userID string,
resultLimit int,
) (domain.CatalogSectionData, error) {
if strings.TrimSpace(userID) == "" {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return domain.CatalogSectionData{}, err
}
now := time.Now()
seedPool := buildRecommendationSeeds(now, watchlist)
if len(seedPool) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
type rankedCandidate struct {
id int
collaborativeScore float64
profileSearchScore float64
anime jikan.Anime
hasAnime bool
}
watchlistAnimeIDs := make(map[int]struct{}, len(watchlist))
for _, entry := range watchlist {
if entry.AnimeID <= 0 {
continue
}
watchlistAnimeIDs[int(entry.AnimeID)] = struct{}{}
}
candidatesByID := map[int]rankedCandidate{}
var candidatesByIDMu sync.Mutex
upsertCandidate := func(candidate rankedCandidate) {
if candidate.id <= 0 {
return
}
if _, exists := watchlistAnimeIDs[candidate.id]; exists {
return
}
candidatesByIDMu.Lock()
defer candidatesByIDMu.Unlock()
current, ok := candidatesByID[candidate.id]
if !ok {
candidatesByID[candidate.id] = candidate
return
}
current.collaborativeScore += candidate.collaborativeScore
current.profileSearchScore += candidate.profileSearchScore
if candidate.hasAnime {
current.anime = candidate.anime
current.hasAnime = true
}
candidatesByID[candidate.id] = current
}
seedAnimes := make([]jikan.Anime, len(seedPool))
var seedFetchGroup errgroup.Group
seedFetchGroup.SetLimit(4)
for i, seed := range seedPool {
seedFetchGroup.Go(func() error {
anime, fetchErr := s.jikan.GetAnimeByID(ctx, seed.animeID)
if fetchErr != nil {
return fetchErr
}
seedAnimes[i] = anime
return nil
})
}
if err := seedFetchGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
profile := buildTasteProfile(now, seedPool, seedAnimes)
var recommendationGroup errgroup.Group
recommendationGroup.SetLimit(4)
for _, seed := range seedPool {
recommendationGroup.Go(func() error {
recs, recErr := s.jikan.GetAnimeRecommendations(ctx, seed.animeID)
if recErr != nil {
return recErr
}
for i, rec := range recs {
if i >= forYouMaxRecommendations {
break
}
id := rec.Entry.MalID
if id <= 0 {
continue
}
if id == seed.animeID {
continue
}
upsertCandidate(rankedCandidate{
id: id,
collaborativeScore: float64(rec.Votes) * seed.weight,
})
}
return nil
})
}
if err := recommendationGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
profileQueries := buildProfileSearchQueries(profile)
var profileSearchGroup errgroup.Group
profileSearchGroup.SetLimit(3)
for _, query := range profileQueries {
profileSearchGroup.Go(func() error {
res, searchErr := s.jikan.SearchAdvanced(
ctx,
"",
"",
"",
"score",
"desc",
query.genreIDs,
query.studioID,
true,
1,
forYouProfileSearchLimit,
)
if searchErr != nil {
observability.Warn(
"top_pick_profile_search_failed",
"anime",
"",
map[string]any{
"genres": query.genreIDs,
"studio_id": query.studioID,
},
searchErr,
)
return nil
}
for i, anime := range res.Animes {
if anime.MalID <= 0 {
continue
}
upsertCandidate(rankedCandidate{
id: anime.MalID,
profileSearchScore: query.weight * profileSearchRankWeight(i),
anime: anime,
hasAnime: true,
})
}
return nil
})
}
if err := profileSearchGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
if len(candidatesByID) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
rankedIDs := make([]rankedCandidate, 0, len(candidatesByID))
for _, item := range candidatesByID {
rankedIDs = append(rankedIDs, item)
}
sort.Slice(rankedIDs, func(i, j int) bool {
left := rankedCandidateRetrievalScore(rankedIDs[i].collaborativeScore, rankedIDs[i].profileSearchScore)
right := rankedCandidateRetrievalScore(rankedIDs[j].collaborativeScore, rankedIDs[j].profileSearchScore)
if left == right {
return rankedIDs[i].id < rankedIDs[j].id
}
return left > right
})
limit := min(len(rankedIDs), forYouCandidateFetchLimit)
candidates := make([]recommendationCandidate, 0, limit)
var candidatesMu sync.Mutex
var detailGroup errgroup.Group
detailGroup.SetLimit(6)
for i := 0; i < limit; i++ {
item := rankedIDs[i]
detailGroup.Go(func() error {
anime := item.anime
if !item.hasAnime || !hasTasteMetadata(anime) {
fetchedAnime, fetchErr := s.jikan.GetAnimeByID(ctx, item.id)
if fetchErr != nil {
observability.Warn(
"recommendation_anime_fetch_failed",
"anime",
"",
map[string]any{"anime_id": item.id},
fetchErr,
)
return nil
}
anime = fetchedAnime
}
candidate := scoreRecommendationCandidate(
now,
profile,
anime,
item.collaborativeScore,
item.profileSearchScore,
)
candidatesMu.Lock()
candidates = append(candidates, candidate)
candidatesMu.Unlock()
return nil
})
}
if err := detailGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].score == candidates[j].score {
return candidates[i].anime.MalID < candidates[j].anime.MalID
}
return candidates[i].score > candidates[j].score
})
return domain.CatalogSectionData{
Animes: rerankRecommendationCandidates(candidates, resultLimit),
}, nil
}
func (s *animeService) GetAiringSchedule(ctx context.Context, userID string) ([]domain.Anime, error) {
if strings.TrimSpace(userID) == "" {
return []domain.Anime{}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return nil, err
}
ids := make([]int, 0, 50)
for _, entry := range watchlist {
status := strings.TrimSpace(entry.Status)
if status != "watching" && status != "plan_to_watch" {
continue
}
if !entry.Airing.Valid || !entry.Airing.Bool {
continue
}
if entry.AnimeID <= 0 {
continue
}
ids = append(ids, int(entry.AnimeID))
if len(ids) >= 50 {
break
}
}
if len(ids) == 0 {
return []domain.Anime{}, nil
}
animes := make([]domain.Anime, 0, len(ids))
var g errgroup.Group
g.SetLimit(6)
var mu sync.Mutex
for _, id := range ids {
g.Go(func() error {
anime, fetchErr := s.jikan.GetAnimeByID(ctx, id)
if fetchErr != nil {
return fetchErr
}
mu.Lock()
animes = append(animes, domain.Anime{Anime: anime})
mu.Unlock()
return nil
})
}
if err := g.Wait(); err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return nil, err
}
observability.Warn(
"schedule_partial_fetch_failed",
"anime",
"",
map[string]any{"user_id": userID, "count": len(ids)},
err,
)
return animes, nil
}
return animes, nil
}
func (s *animeService) GetAnimeByID(ctx context.Context, id int) (domain.Anime, error) {
anime, err := s.jikan.GetAnimeByID(ctx, id)
if err != nil {
@@ -525,8 +172,8 @@ func (s *animeService) GetRecommendations(ctx context.Context, id int) ([]domain
return out, nil
}
func (s *animeService) GetRelations(ctx context.Context, id int) ([]jikan.RelationEntry, error) {
return s.jikan.GetFullRelations(ctx, id)
func (s *animeService) GetRelations(ctx context.Context, id int, mode jikan.WatchOrderMode) ([]jikan.RelationEntry, error) {
return s.jikan.GetFullRelations(ctx, id, mode)
}
func (s *animeService) WarmDetailSections(id int) {

2
internal/app/app.go
View File

@@ -10,6 +10,7 @@ import (
"mal/internal/config"
"mal/internal/database"
"mal/internal/episodes"
"mal/internal/observability"
"mal/internal/playback"
"mal/internal/server"
"mal/internal/watchlist"
@@ -22,6 +23,7 @@ import (
func NewApp() *fx.App {
return fx.New(
fx.WithLogger(observability.NewFxLogger),
config.Module,
database.Module,
audit.Module,

98
internal/audit/service_test.go
View File

@@ -2,6 +2,7 @@ package audit_test
import (
"context"
"database/sql"
"encoding/json"
"os"
"testing"
@@ -13,29 +14,9 @@ import (
)
func TestRecordInsertsAuditLog(t *testing.T) {
tmp, err := os.CreateTemp("", "mal-audit-*.db")
if err != nil {
t.Fatalf("CreateTemp: %v", err)
}
_ = tmp.Close()
t.Cleanup(func() { _ = os.Remove(tmp.Name()) })
sqlDB, err := db.Open(tmp.Name())
if err != nil {
t.Fatalf("db.Open: %v", err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
if err := database.RunMigrations(sqlDB); err != nil {
t.Fatalf("RunMigrations: %v", err)
}
queries := db.New(sqlDB)
svc := audit.NewAuditService(queries)
if _, err := sqlDB.Exec("INSERT INTO user (id, username, password_hash) VALUES (?, ?, ?)", "user-1", "test", "hash"); err != nil {
t.Fatalf("insert user: %v", err)
}
sqlDB := openTestDB(t)
svc := audit.NewAuditService(db.New(sqlDB))
insertTestUser(t, sqlDB, "user-1")
ctx := audit.WithRequestInfo(context.Background(), "127.0.0.1", "unit-test")
metadata, err := json.Marshal(struct {
@@ -55,7 +36,54 @@ func TestRecordInsertsAuditLog(t *testing.T) {
t.Fatalf("Record: %v", err)
}
rows, err := sqlDB.Query("SELECT action, resource_type, resource_id, ip, user_agent, metadata_json FROM audit_log WHERE user_id = ?", "user-1")
auditRow := queryAuditRow(t, sqlDB, "user-1")
assertAuditRow(t, auditRow)
}
type auditRow struct {
action string
resourceType string
resourceID string
ip string
userAgent string
metadataJSON string
}
func openTestDB(t *testing.T) *sql.DB {
t.Helper()
tmp, err := os.CreateTemp("", "mal-audit-*.db")
if err != nil {
t.Fatalf("CreateTemp: %v", err)
}
_ = tmp.Close()
t.Cleanup(func() { _ = os.Remove(tmp.Name()) })
sqlDB, err := db.Open(tmp.Name())
if err != nil {
t.Fatalf("db.Open: %v", err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
if err := database.RunMigrations(sqlDB); err != nil {
t.Fatalf("RunMigrations: %v", err)
}
return sqlDB
}
func insertTestUser(t *testing.T, sqlDB *sql.DB, userID string) {
t.Helper()
if _, err := sqlDB.ExecContext(context.Background(), "INSERT INTO user (id, username, password_hash) VALUES (?, ?, ?)", userID, "test", "hash"); err != nil {
t.Fatalf("insert user: %v", err)
}
}
func queryAuditRow(t *testing.T, sqlDB *sql.DB, userID string) auditRow {
t.Helper()
rows, err := sqlDB.QueryContext(context.Background(), "SELECT action, resource_type, resource_id, ip, user_agent, metadata_json FROM audit_log WHERE user_id = ?", userID)
if err != nil {
t.Fatalf("Query: %v", err)
}
@@ -65,18 +93,24 @@ func TestRecordInsertsAuditLog(t *testing.T) {
t.Fatalf("expected audit row")
}
var action, resourceType, resourceID, ip, userAgent, metadataJSON string
if err := rows.Scan(&action, &resourceType, &resourceID, &ip, &userAgent, &metadataJSON); err != nil {
var row auditRow
if err := rows.Scan(&row.action, &row.resourceType, &row.resourceID, &row.ip, &row.userAgent, &row.metadataJSON); err != nil {
t.Fatalf("Scan: %v", err)
}
if action != "test_action" || resourceType != "thing" || resourceID != "123" {
t.Fatalf("unexpected row action=%q resourceType=%q resourceID=%q", action, resourceType, resourceID)
return row
}
if ip != "127.0.0.1" || userAgent != "unit-test" {
t.Fatalf("unexpected request info ip=%q userAgent=%q", ip, userAgent)
func assertAuditRow(t *testing.T, row auditRow) {
t.Helper()
if row.action != "test_action" || row.resourceType != "thing" || row.resourceID != "123" {
t.Fatalf("unexpected row action=%q resourceType=%q resourceID=%q", row.action, row.resourceType, row.resourceID)
}
if metadataJSON == "" || metadataJSON == "null" {
t.Fatalf("expected metadata_json, got %q", metadataJSON)
if row.ip != "127.0.0.1" || row.userAgent != "unit-test" {
t.Fatalf("unexpected request info ip=%q userAgent=%q", row.ip, row.userAgent)
}
if row.metadataJSON == "" || row.metadataJSON == "null" {
t.Fatalf("expected metadata_json, got %q", row.metadataJSON)
}
}

50
internal/auth/middleware.go
View File

@@ -49,6 +49,33 @@ func isPublicRequest(method string, path string) bool {
return false
}
func authenticateAPIRequest(c *gin.Context, svc domain.AuthService) (*domain.User, string, bool, error) {
authHeader := strings.TrimSpace(c.GetHeader("Authorization"))
if strings.HasPrefix(strings.ToLower(authHeader), "bearer ") {
token := strings.TrimSpace(authHeader[7:])
user, err := svc.ValidateAPIToken(c.Request.Context(), token)
return user, "", false, err
}
sessionID, err := c.Cookie("session_id")
if err != nil {
return nil, "", false, err
}
user, err := svc.ValidateSession(c.Request.Context(), sessionID)
return user, sessionID, true, err
}
func authenticatePageRequest(c *gin.Context, svc domain.AuthService) (*domain.User, string, error) {
sessionID, err := c.Cookie("session_id")
if err != nil {
return nil, "", err
}
user, err := svc.ValidateSession(c.Request.Context(), sessionID)
return user, sessionID, err
}
func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
return func(c *gin.Context) {
path := c.Request.URL.Path
@@ -65,18 +92,7 @@ func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
// API routes can authenticate via Bearer token OR cookie session.
if strings.HasPrefix(path, "/api/") {
authHeader := strings.TrimSpace(c.GetHeader("Authorization"))
if strings.HasPrefix(strings.ToLower(authHeader), "bearer ") {
token := strings.TrimSpace(authHeader[7:])
user, err = svc.ValidateAPIToken(c.Request.Context(), token)
} else if cookieSessionID, cookieErr := c.Cookie("session_id"); cookieErr == nil {
sessionID = cookieSessionID
usesCookieSession = true
user, err = svc.ValidateSession(c.Request.Context(), sessionID)
} else {
err = cookieErr
}
user, sessionID, usesCookieSession, err = authenticateAPIRequest(c, svc)
if err != nil || user == nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "Unauthorized"})
c.Abort()
@@ -84,16 +100,8 @@ func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
}
} else {
// Non-API routes only use cookie sessions and redirect to /login.
cookieSessionID, cookieErr := c.Cookie("session_id")
if cookieErr != nil {
c.Redirect(http.StatusSeeOther, "/login")
c.Abort()
return
}
sessionID = cookieSessionID
user, sessionID, err = authenticatePageRequest(c, svc)
usesCookieSession = true
user, err = svc.ValidateSession(c.Request.Context(), sessionID)
if err != nil || user == nil {
c.Redirect(http.StatusSeeOther, "/login")
c.Abort()

6
internal/auth/module.go
View File

@@ -1,10 +1,8 @@
package auth
import (
"mal/internal/domain"
"mal/internal/server"
"github.com/gin-gonic/gin"
"go.uber.org/fx"
)
@@ -13,9 +11,7 @@ var Module = fx.Options(
NewAuthRepository,
NewAuthService,
NewAuthHandler,
func(svc domain.AuthService) gin.HandlerFunc {
return AuthMiddleware(svc)
},
AuthMiddleware,
),
fx.Provide(
server.AsRouteRegister(func(h *AuthHandler) server.RouteRegister {

5
internal/avatar.go
View File

@@ -6,6 +6,7 @@ import (
)
func DefaultAvatarURL(username string) string {
seed := url.QueryEscape(strings.TrimSpace(username))
return "https://api.dicebear.com/9.x/dylan/svg?seed=" + seed
params := url.Values{}
params.Set("seed", strings.TrimSpace(username))
return "https://api.dicebear.com/9.x/dylan/svg?" + params.Encode()
}

8
internal/database/database.go
View File

@@ -38,6 +38,7 @@ func ProvideQueries(sqlDB *sql.DB) *db.Queries {
func RunMigrations(sqlDB *sql.DB) error {
goose.SetBaseFS(migrationsFS)
goose.SetLogger(goose.NopLogger())
if err := goose.SetDialect("sqlite3"); err != nil {
return fmt.Errorf("failed to set goose dialect: %w", err)
@@ -48,6 +49,13 @@ func RunMigrations(sqlDB *sql.DB) error {
return fmt.Errorf("failed to run migrations: %w", err)
}
version, err := goose.GetDBVersion(sqlDB)
if err != nil {
return fmt.Errorf("failed to get database migration version: %w", err)
}
observability.Info("db_migrations_complete", "database", "", map[string]any{"version": version})
return nil
}
func RunMigrationsAndFixes(sqlDB *sql.DB) error {

3
internal/database/database_test.go
View File

@@ -1,6 +1,7 @@
package database
import (
"context"
"database/sql"
"testing"
@@ -28,7 +29,7 @@ func TestRunMigrationsCreatesHotPathIndexes(t *testing.T) {
} {
t.Run(indexName, func(t *testing.T) {
var count int
err := sqlDB.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type = 'index' AND name = ?`, indexName).Scan(&count)
err := sqlDB.QueryRowContext(context.Background(), `SELECT COUNT(*) FROM sqlite_master WHERE type = 'index' AND name = ?`, indexName).Scan(&count)
if err != nil {
t.Fatalf("query index: %v", err)
}

90
internal/database/fixes/20260608_backfill_anime_duration_seconds.go
View File

@@ -10,50 +10,25 @@ import (
"mal/internal/observability"
)
func init() {
Register(Fix{
ID: "20260608_backfill_anime_duration_seconds",
Apply: func(ctx context.Context, sqlDB *sql.DB) error {
rows, err := sqlDB.QueryContext(ctx, `
SELECT id, title_original, title_english, title_japanese, image_url, airing
FROM anime
WHERE duration_seconds IS NULL;
`)
if err != nil {
return fmt.Errorf("query anime rows missing duration_seconds: %w", err)
}
defer func() { _ = rows.Close() }()
client := jikan.NewClient(config.Config{}, db.New(sqlDB), observability.NewMetrics())
type animeRow struct {
type animeDurationRow struct {
id int64
titleOriginal string
}
var toUpdate []animeRow
for rows.Next() {
var row animeRow
var titleEnglish sql.NullString
var titleJapanese sql.NullString
var imageURL string
var airing sql.NullBool
if err := rows.Scan(
&row.id,
&row.titleOriginal,
&titleEnglish,
&titleJapanese,
&imageURL,
&airing,
); err != nil {
return fmt.Errorf("scan anime row missing duration_seconds: %w", err)
}
toUpdate = append(toUpdate, row)
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate anime rows missing duration_seconds: %w", err)
func init() {
Register(Fix{
ID: "20260608_backfill_anime_duration_seconds",
Apply: applyAnimeDurationSecondsBackfill,
})
}
func applyAnimeDurationSecondsBackfill(ctx context.Context, sqlDB *sql.DB) error {
toUpdate, err := listAnimeMissingDurationSeconds(ctx, sqlDB)
if err != nil {
return err
}
client := jikan.NewClient(config.Config{}, db.New(sqlDB), observability.NewMetrics())
for _, row := range toUpdate {
anime, err := client.GetAnimeByID(ctx, int(row.id))
if err != nil {
@@ -76,6 +51,41 @@ WHERE duration_seconds IS NULL;
}
return nil
},
})
}
func listAnimeMissingDurationSeconds(ctx context.Context, sqlDB *sql.DB) ([]animeDurationRow, error) {
rows, err := sqlDB.QueryContext(ctx, `
SELECT id, title_original, title_english, title_japanese, image_url, airing
FROM anime
WHERE duration_seconds IS NULL;
`)
if err != nil {
return nil, fmt.Errorf("query anime rows missing duration_seconds: %w", err)
}
defer func() { _ = rows.Close() }()
var toUpdate []animeDurationRow
for rows.Next() {
var row animeDurationRow
var titleEnglish sql.NullString
var titleJapanese sql.NullString
var imageURL string
var airing sql.NullBool
if err := rows.Scan(
&row.id,
&row.titleOriginal,
&titleEnglish,
&titleJapanese,
&imageURL,
&airing,
); err != nil {
return nil, fmt.Errorf("scan anime row missing duration_seconds: %w", err)
}
toUpdate = append(toUpdate, row)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate anime rows missing duration_seconds: %w", err)
}
return toUpdate, nil
}

76
internal/db/command_palette.go
View File

@@ -9,9 +9,7 @@ func (q *Queries) GetCommandPaletteContinueWatching(ctx context.Context, userID
if userID == "" {
return nil, nil
}
if limit <= 0 {
limit = 5
}
limit = commandPaletteLimit(limit)
needle, pattern := commandPalettePattern(query)
rows, err := q.db.QueryContext(ctx, `
@@ -48,8 +46,22 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
items := make([]GetContinueWatchingEntriesRow, 0, int(limit))
for rows.Next() {
item, err := scanContinueWatchingEntry(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
func scanContinueWatchingEntry(rows scanner) (GetContinueWatchingEntriesRow, error) {
var item GetContinueWatchingEntriesRow
if err := rows.Scan(
err := rows.Scan(
&item.ID,
&item.UserID,
&item.AnimeID,
@@ -63,25 +75,15 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
&item.TitleJapanese,
&item.ImageUrl,
&item.AnimeDurationSeconds,
); err != nil {
return nil, err
}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
)
return item, err
}
func (q *Queries) GetCommandPaletteWatchlist(ctx context.Context, userID string, query string, limit int64) ([]GetUserWatchListRow, error) {
if userID == "" {
return nil, nil
}
if limit <= 0 {
limit = 5
}
limit = commandPaletteLimit(limit)
needle, pattern := commandPalettePattern(query)
rows, err := q.db.QueryContext(ctx, `
@@ -126,8 +128,22 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
items := make([]GetUserWatchListRow, 0, int(limit))
for rows.Next() {
item, err := scanWatchListEntry(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
func scanWatchListEntry(rows scanner) (GetUserWatchListRow, error) {
var item GetUserWatchListRow
if err := rows.Scan(
err := rows.Scan(
&item.ID,
&item.UserID,
&item.AnimeID,
@@ -142,19 +158,23 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
&item.TitleJapanese,
&item.ImageUrl,
&item.Airing,
); err != nil {
return nil, err
}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
)
return item, err
}
func commandPalettePattern(query string) (string, string) {
needle := strings.ToLower(strings.TrimSpace(query))
return needle, "%" + needle + "%"
}
func commandPaletteLimit(limit int64) int64 {
if limit <= 0 {
return 5
}
return limit
}
type scanner interface {
Scan(dest ...interface{}) error
}

2
internal/db/command_palette_test.go
View File

@@ -60,7 +60,7 @@ func openCommandPaletteTestDB(t *testing.T) *sql.DB {
}
t.Cleanup(func() { _ = sqlDB.Close() })
_, err = sqlDB.Exec(`
_, err = sqlDB.ExecContext(context.Background(), `
CREATE TABLE anime (
id INTEGER PRIMARY KEY,
title_original TEXT NOT NULL,

10
internal/db/helpers.go
View File

@@ -11,9 +11,15 @@ func NullStringOr(n sql.NullString, fallback string) string {
return fallback
}
// DisplayTitle returns the English title, falling back to Japanese then original
// DisplayTitle returns the English title, falling back to original then Japanese.
func DisplayTitle(titleEnglish, titleJapanese sql.NullString, titleOriginal string) string {
return NullStringOr(titleEnglish, NullStringOr(titleJapanese, titleOriginal))
if titleEnglish.Valid && titleEnglish.String != "" {
return titleEnglish.String
}
if titleOriginal != "" {
return titleOriginal
}
return NullStringOr(titleJapanese, titleOriginal)
}
func (r GetUserWatchListRow) DisplayTitle() string {

30
internal/db/helpers_test.go Normal file
View File

@@ -0,0 +1,30 @@
package db
import (
"database/sql"
"testing"
)
func TestDisplayTitlePrefersOriginalBeforeJapanese(t *testing.T) {
got := DisplayTitle(
sql.NullString{},
sql.NullString{String: "サイバーパンク エッジランナーズ", Valid: true},
"Cyberpunk: Edgerunners",
)
if got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want original title", got)
}
}
func TestDisplayTitlePrefersEnglish(t *testing.T) {
got := DisplayTitle(
sql.NullString{String: "Frieren: Beyond Journey's End", Valid: true},
sql.NullString{String: "葬送のフリーレン", Valid: true},
"Sousou no Frieren",
)
if got != "Frieren: Beyond Journey's End" {
t.Fatalf("DisplayTitle() = %q, want English title", got)
}
}

3
internal/db/skip_segment_overrides.go
View File

@@ -7,9 +7,6 @@ import (
"fmt"
)
// Note: we intentionally avoid naming this struct SkipSegmentOverride because
// some environments may have an sqlc-generated SkipSegmentOverride model,
// which would cause a redeclare build error.
type SkipSegmentOverrideRow struct {
ID string
UserID string

5
internal/db/sqlite.go
View File

@@ -1,6 +1,7 @@
package db
import (
"context"
"database/sql"
"fmt"
@@ -17,7 +18,7 @@ func Open(dbFile string) (*sql.DB, error) {
return nil, fmt.Errorf("failed to open db: %w", err)
}
// WAL improves concurrency between readers and writers.
_, _ = db.Exec("PRAGMA journal_mode=WAL;")
_, _ = db.Exec("PRAGMA busy_timeout=5000;")
_, _ = db.ExecContext(context.Background(), "PRAGMA journal_mode=WAL;")
_, _ = db.ExecContext(context.Background(), "PRAGMA busy_timeout=5000;")
return db, nil
}

2
internal/db/watchlist_ids_test.go
View File

@@ -16,7 +16,7 @@ func TestGetUserWatchlistAnimeIDsFiltersRequestedIDs(t *testing.T) {
}
defer func() { _ = sqlDB.Close() }()
_, err = sqlDB.Exec(`
_, err = sqlDB.ExecContext(context.Background(), `
CREATE TABLE watch_list_entry (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,

18
internal/domain/anime.go
View File

@@ -137,11 +137,6 @@ type AnimeCatalogService interface {
GetTopPicksForYou(ctx context.Context, userID string) (CatalogSectionData, error)
}
type AnimeDiscoverService interface {
GetDiscoverSection(ctx context.Context, userID string, section string) (DiscoverSectionData, error)
GetAiringSchedule(ctx context.Context, userID string) ([]Anime, error)
}
type AnimeSearchService interface {
SearchAdvanced(ctx context.Context, q, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (jikan.SearchResult, error)
GetProducerNameByID(ctx context.Context, id int) (string, error)
@@ -153,7 +148,7 @@ type AnimeDetailsService interface {
GetAnimeByID(ctx context.Context, id int) (Anime, error)
GetCharacters(ctx context.Context, id int) ([]CharacterEntry, error)
GetRecommendations(ctx context.Context, id int) ([]RecommendationEntry, error)
GetRelations(ctx context.Context, id int) ([]jikan.RelationEntry, error)
GetRelations(ctx context.Context, id int, mode jikan.WatchOrderMode) ([]jikan.RelationEntry, error)
GetEpisodes(ctx context.Context, id int, page int) (jikan.EpisodesResponse, error)
GetAllEpisodes(ctx context.Context, id int) ([]EpisodeData, error)
GetRandomAnime(ctx context.Context) (Anime, error)
@@ -180,17 +175,6 @@ func (d CatalogSectionData) TemplateFragment() string {
return d.Fragment
}
type DiscoverSectionData struct {
Animes []Anime
Section string
WatchlistMap map[int64]bool
Fragment string
}
func (d DiscoverSectionData) TemplateFragment() string {
return d.Fragment
}
type AnimeRepository interface {
GetUserWatchList(ctx context.Context, userID string) ([]db.GetUserWatchListRow, error)
GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error)

3
internal/domain/playback.go
View File

@@ -49,6 +49,7 @@ type SubtitleItem struct {
type ModeSource struct {
Token string `json:"token"`
Type string `json:"type,omitempty"`
Subtitles []SubtitleItem `json:"subtitles"`
Qualities []string `json:"qualities,omitempty"`
}
@@ -88,6 +89,8 @@ type EpisodeData struct {
type PlaybackRepository interface {
InTx(ctx context.Context, fn func(ctx context.Context, repo PlaybackRepository) error) error
UpsertAnime(ctx context.Context, params db.UpsertAnimeParams) (db.Anime, error)
GetAnime(ctx context.Context, id int64) (db.Anime, error)
GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error)
GetContinueWatchingEntry(ctx context.Context, params db.GetContinueWatchingEntryParams) (db.ContinueWatchingEntry, error)
SaveWatchProgress(ctx context.Context, params db.SaveWatchProgressParams) error

1
internal/domain/provider.go
View File

@@ -12,6 +12,7 @@ type StreamSource struct {
type StreamResult struct {
URL string
Referer string
Type string
Subtitles []Subtitle
Qualities []StreamSource
}

7
internal/episodes/module.go
View File

@@ -2,13 +2,10 @@
package episodes
import (
"mal/integrations/jikan"
"mal/integrations/playback/allanime"
"mal/internal/config"
"mal/internal/db"
"mal/internal/domain"
episodeService "mal/internal/episodes/service"
"mal/internal/observability"
"go.uber.org/fx"
)
@@ -21,9 +18,7 @@ var Module = fx.Options(
fx.Provide(
episodeAvailabilityEnabled,
fx.Annotate(
func(queries *db.Queries, jikanClient *jikan.Client, providers []domain.EpisodeAvailabilityProvider, enabled bool, metrics *observability.Metrics) domain.EpisodeService {
return episodeService.NewEpisodeService(queries, jikanClient, providers, enabled, metrics)
},
episodeService.NewEpisodeService,
),
),
fx.Provide(func(p *allanime.AllAnimeProvider) []domain.EpisodeAvailabilityProvider {

250
internal/episodes/service/cache_store.go Normal file
View File

@@ -0,0 +1,250 @@
package service
import (
"context"
"database/sql"
"encoding/json"
"time"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *EpisodeService) store(ctx context.Context, anime domain.Anime, jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, source string, now time.Time, providerSuccess bool) (domain.CanonicalEpisodeList, error) {
nextRefreshSQL := nextRefreshAt(anime, now)
episodes := mergeEpisodes(jikanEpisodes, availability, anime.Episodes)
payload := domain.CanonicalEpisodeList{
AnimeID: anime.MalID,
Episodes: episodes,
Source: source,
}
if nextRefreshSQL.Valid {
payload.NextRefreshAt = nextRefreshSQL.Time.Format(time.RFC3339)
}
body, err := json.Marshal(payload)
if err != nil {
return domain.CanonicalEpisodeList{}, err
}
if !s.writeEpisodeAvailabilityCache(ctx, anime, source, body, now, providerSuccess, nextRefreshSQL) {
return payload, nil
}
observability.Info(
"episodes_refresh_success",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
"episodes": len(episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, nil
}
func (s *EpisodeService) writeEpisodeAvailabilityCache(ctx context.Context, anime domain.Anime, source string, body []byte, now time.Time, providerSuccess bool, nextRefreshSQL sql.NullTime) bool {
var retryUntil sql.NullTime
if anime.Airing && providerSuccess {
retryUntil = sql.NullTime{Time: nextRefreshSQL.Time.Add(retryWindow), Valid: nextRefreshSQL.Valid}
}
err := s.queries.UpsertEpisodeAvailabilityCache(ctx, db.UpsertEpisodeAvailabilityCacheParams{
AnimeID: int64(anime.MalID),
Data: string(body),
NextRefreshAt: nextRefreshSQL,
RetryUntilAt: retryUntil,
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastSuccessAt: sql.NullTime{Time: now, Valid: providerSuccess},
FailureCount: 0,
LastError: "",
})
if err == nil {
return true
}
observability.Warn(
"episodes_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
},
err,
)
return false
}
func (s *EpisodeService) markFailure(ctx context.Context, anime domain.Anime, cause error) {
now := s.clock.Now()
next := nextRetryTime(anime, now)
var retryUntil sql.NullTime
nextBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !nextBroadcast.IsZero() {
retryUntil = sql.NullTime{Time: nextBroadcast.Add(retryWindow), Valid: true}
}
var nextSQL sql.NullTime
if !next.IsZero() {
nextSQL = sql.NullTime{Time: next, Valid: true}
}
writeCtx := ctx
if ctx.Err() != nil {
var cancel context.CancelFunc
writeCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
err := s.queries.MarkEpisodeAvailabilityRefreshFailed(writeCtx, db.MarkEpisodeAvailabilityRefreshFailedParams{
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastError: truncate(cause.Error(), 400),
NextRefreshAt: nextSQL,
RetryUntilAt: retryUntil,
AnimeID: int64(anime.MalID),
})
if err != nil {
observability.Warn(
"episodes_mark_failure_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return
}
observability.Warn(
"episodes_refresh_failure_recorded",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_retry": next.Format(time.RFC3339),
},
cause,
)
}
func (s *EpisodeService) getCached(ctx context.Context, animeID int) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(animeID))
if err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": animeID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability", "hit")
return payload, true
}
func (s *EpisodeService) getFreshCached(ctx context.Context, anime domain.Anime) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(anime.MalID))
if err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
return domain.CanonicalEpisodeList{}, false
}
now := s.clock.Now()
if !s.isFreshEpisodeCache(anime, row, now) {
return domain.CanonicalEpisodeList{}, false
}
payload, ok := s.decodeFreshCachedPayload(anime, row.Data)
if !ok {
return domain.CanonicalEpisodeList{}, false
}
if !isCanonicalEpisodePayloadValid(payload, anime.Episodes) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cached_payload_rejected",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"expected_count": anime.Episodes,
"cached_episodes": len(payload.Episodes),
},
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability_fresh", "hit")
observability.Info(
"episodes_cache_served",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"episodes": len(payload.Episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, true
}
func (s *EpisodeService) isFreshEpisodeCache(anime domain.Anime, row db.EpisodeAvailabilityCache, now time.Time) bool {
if row.NextRefreshAt.Valid && !row.NextRefreshAt.Time.After(now) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_due_for_refresh",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_refresh": row.NextRefreshAt.Time.Format(time.RFC3339),
},
)
return false
}
if anime.Airing && row.UpdatedAt.Before(now.Add(-airingFallbackRefreshInterval)) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_too_old_for_airing",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"updated_at": row.UpdatedAt.Format(time.RFC3339),
},
)
return false
}
return true
}
func (s *EpisodeService) decodeFreshCachedPayload(anime domain.Anime, raw string) (domain.CanonicalEpisodeList, bool) {
var payload domain.CanonicalEpisodeList
err := json.Unmarshal([]byte(raw), &payload)
if err == nil {
return payload, true
}
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}

133
internal/episodes/service/merge.go Normal file
View File

@@ -0,0 +1,133 @@
package service
import (
"fmt"
"sort"
"strconv"
"strings"
"mal/integrations/jikan"
"mal/internal/domain"
)
type episodePartial struct {
title string
filler bool
recap bool
sub bool
dub bool
}
func titleCandidates(anime domain.Anime) []string {
out := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
out = append(out, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
out = append(out, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
out = append(out, syn)
}
}
return out
}
func isCanonicalEpisodePayloadValid(payload domain.CanonicalEpisodeList, expectedCount int) bool {
if expectedCount <= 0 {
return true
}
if len(payload.Episodes) > expectedCount {
return false
}
for _, episode := range payload.Episodes {
if episode.Number <= 0 || episode.Number > expectedCount {
return false
}
}
return true
}
func mergeEpisodes(jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, expectedCount int) []domain.CanonicalEpisode {
byNumber := map[int]episodePartial{}
for i, ep := range jikanEpisodes {
if exceedsExpectedCount(i+1, expectedCount) {
break
}
number, ok := jikanEpisodeNumber(ep, i)
if !ok || exceedsExpectedCount(number, expectedCount) {
continue
}
mergeEpisode(&byNumber, number, func(item *episodePartial) {
item.title = strings.TrimSpace(ep.Title)
item.filler = ep.Filler
item.recap = ep.Recap
})
}
mergeAvailability(&byNumber, availability.Sub, expectedCount, func(item *episodePartial) { item.sub = true })
mergeAvailability(&byNumber, availability.Dub, expectedCount, func(item *episodePartial) { item.dub = true })
numbers := make([]int, 0, len(byNumber))
for number := range byNumber {
numbers = append(numbers, number)
}
sort.Ints(numbers)
episodes := make([]domain.CanonicalEpisode, 0, len(numbers))
for _, number := range numbers {
item := byNumber[number]
title := item.title
if title == "" {
title = fmt.Sprintf("Episode %d", number)
}
episodes = append(episodes, domain.CanonicalEpisode{
Number: number,
Title: title,
HasSub: item.sub,
HasDub: item.dub,
SubOnly: item.sub && !item.dub,
Filler: item.filler,
Recap: item.recap,
})
}
return episodes
}
func mergeEpisode(byNumber *map[int]episodePartial, number int, update func(*episodePartial)) {
item := (*byNumber)[number]
update(&item)
(*byNumber)[number] = item
}
func mergeAvailability(byNumber *map[int]episodePartial, numbers []int, expectedCount int, update func(*episodePartial)) {
for _, number := range numbers {
if number <= 0 || exceedsExpectedCount(number, expectedCount) {
continue
}
mergeEpisode(byNumber, number, update)
}
}
func jikanEpisodeNumber(ep jikan.Episode, index int) (int, bool) {
number, err := strconv.Atoi(strings.TrimSpace(ep.Episode))
if err == nil && number > 0 {
return number, true
}
if index < 0 {
return 0, false
}
return index + 1, true
}
func exceedsExpectedCount(number int, expectedCount int) bool {
return expectedCount > 0 && number > expectedCount
}
func truncate(value string, maxLen int) string {
if len(value) <= maxLen {
return value
}
return value[:maxLen]
}

120
internal/episodes/service/provider_mapping.go Normal file
View File

@@ -0,0 +1,120 @@
package service
import (
"context"
"database/sql"
"errors"
"fmt"
"strings"
"time"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *EpisodeService) providerID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, titles []string) (string, error) {
providerID, found, err := s.cachedProviderID(ctx, anime, provider)
if found || err != nil {
return providerID, err
}
providerID, err = provider.ResolveEpisodeProviderID(ctx, anime.MalID, titles)
if err != nil {
s.cacheProviderIDFailure(ctx, anime, provider, err)
return "", err
}
s.cacheProviderIDSuccess(ctx, anime, provider, providerID)
observability.Info(
"episodes_provider_id_resolved",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": providerID,
},
)
return providerID, nil
}
func (s *EpisodeService) cachedProviderID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider) (string, bool, error) {
row, err := s.queries.GetEpisodeProviderMapping(ctx, db.GetEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
})
if err != nil {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
if errors.Is(err, sql.ErrNoRows) {
return "", false, nil
}
observability.Warn(
"episodes_provider_id_cache_read_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
return "", false, nil
}
if row.FailedUntil.Valid && row.FailedUntil.Time.After(s.clock.Now()) {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
return "", true, fmt.Errorf("cached provider mapping failure active until %s: %s", row.FailedUntil.Time.Format(time.RFC3339), row.LastError)
}
if strings.TrimSpace(row.ProviderShowID) == "" {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
return "", false, nil
}
s.metrics.ObserveCache("episode_provider_mapping", "hit")
observability.Info(
"episodes_provider_id_cache_hit",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": row.ProviderShowID,
},
)
return row.ProviderShowID, true, nil
}
func (s *EpisodeService) cacheProviderIDFailure(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, resolveErr error) {
_ = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: "",
FailedUntil: sql.NullTime{Time: s.clock.Now().Add(time.Hour), Valid: true},
LastError: truncate(resolveErr.Error(), 400),
})
}
func (s *EpisodeService) cacheProviderIDSuccess(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, providerID string) {
err := s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: providerID,
FailedUntil: sql.NullTime{},
LastError: "",
})
if err == nil {
return
}
observability.Warn(
"episodes_provider_id_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
}

131
internal/episodes/service/refresh_policy.go Normal file
View File

@@ -0,0 +1,131 @@
package service
import (
"database/sql"
"strings"
"time"
"mal/internal/domain"
"mal/internal/observability"
)
const (
retryInterval = 15 * time.Minute
retryWindow = 3 * time.Hour
airingFallbackRefreshInterval = 6 * time.Hour
)
func nextRefreshAt(anime domain.Anime, now time.Time) sql.NullTime {
if !anime.Airing {
return sql.NullTime{}
}
// During the hours immediately following a broadcast time, providers can lag.
// Keep retrying for a short window, even if the provider request succeeded.
lastBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !lastBroadcast.IsZero() && now.Before(lastBroadcast.Add(retryWindow)) {
return sql.NullTime{Time: now.Add(retryInterval).UTC(), Valid: true}
}
next := nextBroadcastAfter(anime, now)
if !next.IsZero() {
return sql.NullTime{Time: next, Valid: true}
}
// Broadcast metadata is often missing or wrong for currently airing shows.
// Avoid "never refresh again" caches by falling back to a fixed interval.
return sql.NullTime{Time: now.Add(airingFallbackRefreshInterval).UTC(), Valid: true}
}
func nextRetryTime(anime domain.Anime, now time.Time) time.Time {
broadcast := nextBroadcastBeforeOrAt(anime, now)
if broadcast.IsZero() || now.After(broadcast.Add(retryWindow)) {
return nextBroadcastAfter(anime, now)
}
return now.Add(retryInterval)
}
func nextBroadcastBeforeOrAt(anime domain.Anime, now time.Time) time.Time {
next := nextBroadcastAfter(anime, now.AddDate(0, 0, -7))
if next.IsZero() || next.After(now) {
return time.Time{}
}
return next
}
func nextBroadcastAfter(anime domain.Anime, after time.Time) time.Time {
day := weekdayFromJikan(anime.Broadcast.Day)
if day < 0 || strings.TrimSpace(anime.Broadcast.Time) == "" {
return time.Time{}
}
loc := time.UTC
if tz := strings.TrimSpace(anime.Broadcast.Timezone); tz != "" {
if loaded, err := time.LoadLocation(tz); err == nil {
loc = loaded
} else {
observability.Warn(
"episodes_broadcast_timezone_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"timezone": tz,
},
err,
)
}
}
hour, minute, ok := parseBroadcastTime(anime.Broadcast.Time)
if !ok {
observability.Warn(
"episodes_broadcast_time_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"time": anime.Broadcast.Time,
},
nil,
)
return time.Time{}
}
localAfter := after.In(loc)
daysAhead := (int(day) - int(localAfter.Weekday()) + 7) % 7
candidate := time.Date(localAfter.Year(), localAfter.Month(), localAfter.Day()+daysAhead, hour, minute, 0, 0, loc)
if !candidate.After(localAfter) {
candidate = candidate.AddDate(0, 0, 7)
}
return candidate.UTC()
}
func weekdayFromJikan(day string) time.Weekday {
switch strings.ToLower(strings.TrimSpace(day)) {
case "sundays":
return time.Sunday
case "mondays":
return time.Monday
case "tuesdays":
return time.Tuesday
case "wednesdays":
return time.Wednesday
case "thursdays":
return time.Thursday
case "fridays":
return time.Friday
case "saturdays":
return time.Saturday
default:
return -1
}
}
func parseBroadcastTime(value string) (int, int, bool) {
t, err := time.Parse("15:04", strings.TrimSpace(value))
if err != nil {
return 0, 0, false
}
return t.Hour(), t.Minute(), true
}

544
internal/episodes/service/service.go
View File

@@ -3,24 +3,13 @@ package service
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
"sort"
"strconv"
"strings"
"time"
)
const (
retryInterval = 15 * time.Minute
retryWindow = 3 * time.Hour
airingFallbackRefreshInterval = 6 * time.Hour
)
type Clock interface {
@@ -229,337 +218,6 @@ func (s *EpisodeService) fetchProviderAvailability(ctx context.Context, anime do
return domain.EpisodeAvailability{}, "", fmt.Errorf("no episode availability provider matched anime_id=%d", anime.MalID)
}
func (s *EpisodeService) providerID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, titles []string) (string, error) {
row, err := s.queries.GetEpisodeProviderMapping(ctx, db.GetEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
})
if err == nil {
if row.FailedUntil.Valid && row.FailedUntil.Time.After(s.clock.Now()) {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
return "", fmt.Errorf("cached provider mapping failure active until %s: %s", row.FailedUntil.Time.Format(time.RFC3339), row.LastError)
}
if strings.TrimSpace(row.ProviderShowID) != "" {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
observability.Info(
"episodes_provider_id_cache_hit",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": row.ProviderShowID,
},
)
return row.ProviderShowID, nil
}
s.metrics.ObserveCache("episode_provider_mapping", "miss")
} else if !errors.Is(err, sql.ErrNoRows) {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
observability.Warn(
"episodes_provider_id_cache_read_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
} else {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
}
providerID, err := provider.ResolveEpisodeProviderID(ctx, anime.MalID, titles)
if err != nil {
_ = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: "",
FailedUntil: sql.NullTime{Time: s.clock.Now().Add(time.Hour), Valid: true},
LastError: truncate(err.Error(), 400),
})
return "", err
}
err = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: providerID,
FailedUntil: sql.NullTime{},
LastError: "",
})
if err != nil {
observability.Warn(
"episodes_provider_id_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
}
observability.Info(
"episodes_provider_id_resolved",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": providerID,
},
)
return providerID, nil
}
func (s *EpisodeService) store(ctx context.Context, anime domain.Anime, jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, source string, now time.Time, providerSuccess bool) (domain.CanonicalEpisodeList, error) {
var nextRefreshSQL sql.NullTime
if anime.Airing {
// During the hours immediately following a broadcast time, providers can lag.
// Keep retrying for a short window, even if the provider request succeeded.
lastBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !lastBroadcast.IsZero() && now.Before(lastBroadcast.Add(retryWindow)) {
nextRefreshSQL = sql.NullTime{Time: now.Add(retryInterval).UTC(), Valid: true}
} else {
next := nextBroadcastAfter(anime, now)
if !next.IsZero() {
nextRefreshSQL = sql.NullTime{Time: next, Valid: true}
} else {
// Broadcast metadata is often missing or wrong for currently airing shows.
// Avoid "never refresh again" caches by falling back to a fixed interval.
nextRefreshSQL = sql.NullTime{Time: now.Add(airingFallbackRefreshInterval).UTC(), Valid: true}
}
}
}
episodes := mergeEpisodes(jikanEpisodes, availability, anime.Episodes)
payload := domain.CanonicalEpisodeList{
AnimeID: anime.MalID,
Episodes: episodes,
Source: source,
}
if nextRefreshSQL.Valid {
payload.NextRefreshAt = nextRefreshSQL.Time.Format(time.RFC3339)
}
body, err := json.Marshal(payload)
if err != nil {
return domain.CanonicalEpisodeList{}, err
}
var retryUntil sql.NullTime
if anime.Airing && providerSuccess {
retryUntil = sql.NullTime{Time: nextRefreshSQL.Time.Add(retryWindow), Valid: nextRefreshSQL.Valid}
}
err = s.queries.UpsertEpisodeAvailabilityCache(ctx, db.UpsertEpisodeAvailabilityCacheParams{
AnimeID: int64(anime.MalID),
Data: string(body),
NextRefreshAt: nextRefreshSQL,
RetryUntilAt: retryUntil,
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastSuccessAt: sql.NullTime{Time: now, Valid: providerSuccess},
FailureCount: 0,
LastError: "",
})
if err != nil {
observability.Warn(
"episodes_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
},
err,
)
return payload, nil
}
observability.Info(
"episodes_refresh_success",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
"episodes": len(episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, nil
}
func (s *EpisodeService) markFailure(ctx context.Context, anime domain.Anime, cause error) {
now := s.clock.Now()
next := nextRetryTime(anime, now)
var retryUntil sql.NullTime
nextBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !nextBroadcast.IsZero() {
retryUntil = sql.NullTime{Time: nextBroadcast.Add(retryWindow), Valid: true}
}
var nextSQL sql.NullTime
if !next.IsZero() {
nextSQL = sql.NullTime{Time: next, Valid: true}
}
writeCtx := ctx
if ctx.Err() != nil {
var cancel context.CancelFunc
writeCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
err := s.queries.MarkEpisodeAvailabilityRefreshFailed(writeCtx, db.MarkEpisodeAvailabilityRefreshFailedParams{
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastError: truncate(cause.Error(), 400),
NextRefreshAt: nextSQL,
RetryUntilAt: retryUntil,
AnimeID: int64(anime.MalID),
})
if err != nil {
observability.Warn(
"episodes_mark_failure_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return
}
observability.Warn(
"episodes_refresh_failure_recorded",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_retry": next.Format(time.RFC3339),
},
cause,
)
}
func (s *EpisodeService) getCached(ctx context.Context, animeID int) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(animeID))
if err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": animeID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability", "hit")
return payload, true
}
func (s *EpisodeService) getFreshCached(ctx context.Context, anime domain.Anime) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(anime.MalID))
if err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
return domain.CanonicalEpisodeList{}, false
}
now := s.clock.Now()
if row.NextRefreshAt.Valid && !row.NextRefreshAt.Time.After(now) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_due_for_refresh",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_refresh": row.NextRefreshAt.Time.Format(time.RFC3339),
},
)
return domain.CanonicalEpisodeList{}, false
}
if anime.Airing && row.UpdatedAt.Before(now.Add(-airingFallbackRefreshInterval)) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_too_old_for_airing",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"updated_at": row.UpdatedAt.Format(time.RFC3339),
},
)
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
if !isCanonicalEpisodePayloadValid(payload, anime.Episodes) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cached_payload_rejected",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"expected_count": anime.Episodes,
"cached_episodes": len(payload.Episodes),
},
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability_fresh", "hit")
observability.Info(
"episodes_cache_served",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"episodes": len(payload.Episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, true
}
func isCanonicalEpisodePayloadValid(payload domain.CanonicalEpisodeList, expectedCount int) bool {
if expectedCount <= 0 {
return true
}
if len(payload.Episodes) > expectedCount {
return false
}
for _, episode := range payload.Episodes {
if episode.Number <= 0 || episode.Number > expectedCount {
return false
}
}
return true
}
func (s *EpisodeService) jikanOnly(ctx context.Context, anime domain.Anime, source string) (domain.CanonicalEpisodeList, error) {
episodes, err := s.jikan.GetAllEpisodes(ctx, anime.MalID)
if err != nil {
@@ -571,201 +229,3 @@ func (s *EpisodeService) jikanOnly(ctx context.Context, anime domain.Anime, sour
Source: source,
}, nil
}
func titleCandidates(anime domain.Anime) []string {
out := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
out = append(out, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
out = append(out, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
out = append(out, syn)
}
}
return out
}
func mergeEpisodes(jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, expectedCount int) []domain.CanonicalEpisode {
type partial struct {
title string
filler bool
recap bool
sub bool
dub bool
}
byNumber := map[int]partial{}
for i, ep := range jikanEpisodes {
if expectedCount > 0 && i >= expectedCount {
break
}
number, ok := jikanEpisodeNumber(ep, i)
if !ok || exceedsExpectedCount(number, expectedCount) {
continue
}
item := byNumber[number]
item.title = strings.TrimSpace(ep.Title)
item.filler = ep.Filler
item.recap = ep.Recap
byNumber[number] = item
}
for _, n := range availability.Sub {
if n <= 0 || exceedsExpectedCount(n, expectedCount) {
continue
}
item := byNumber[n]
item.sub = true
byNumber[n] = item
}
for _, n := range availability.Dub {
if n <= 0 || exceedsExpectedCount(n, expectedCount) {
continue
}
item := byNumber[n]
item.dub = true
byNumber[n] = item
}
numbers := make([]int, 0, len(byNumber))
for number := range byNumber {
numbers = append(numbers, number)
}
sort.Ints(numbers)
episodes := make([]domain.CanonicalEpisode, 0, len(numbers))
for _, number := range numbers {
item := byNumber[number]
title := item.title
if title == "" {
title = fmt.Sprintf("Episode %d", number)
}
episodes = append(episodes, domain.CanonicalEpisode{
Number: number,
Title: title,
HasSub: item.sub,
HasDub: item.dub,
SubOnly: item.sub && !item.dub,
Filler: item.filler,
Recap: item.recap,
})
}
return episodes
}
func jikanEpisodeNumber(ep jikan.Episode, index int) (int, bool) {
number, err := strconv.Atoi(strings.TrimSpace(ep.Episode))
if err == nil && number > 0 {
return number, true
}
if index < 0 {
return 0, false
}
return index + 1, true
}
func exceedsExpectedCount(number int, expectedCount int) bool {
return expectedCount > 0 && number > expectedCount
}
func nextRetryTime(anime domain.Anime, now time.Time) time.Time {
broadcast := nextBroadcastBeforeOrAt(anime, now)
if broadcast.IsZero() || now.After(broadcast.Add(retryWindow)) {
return nextBroadcastAfter(anime, now)
}
return now.Add(retryInterval)
}
func nextBroadcastBeforeOrAt(anime domain.Anime, now time.Time) time.Time {
next := nextBroadcastAfter(anime, now.AddDate(0, 0, -7))
if next.IsZero() || next.After(now) {
return time.Time{}
}
return next
}
func nextBroadcastAfter(anime domain.Anime, after time.Time) time.Time {
day := weekdayFromJikan(anime.Broadcast.Day)
if day < 0 || strings.TrimSpace(anime.Broadcast.Time) == "" {
return time.Time{}
}
loc := time.UTC
if tz := strings.TrimSpace(anime.Broadcast.Timezone); tz != "" {
if loaded, err := time.LoadLocation(tz); err == nil {
loc = loaded
} else {
observability.Warn(
"episodes_broadcast_timezone_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"timezone": tz,
},
err,
)
}
}
hour, minute, ok := parseBroadcastTime(anime.Broadcast.Time)
if !ok {
observability.Warn(
"episodes_broadcast_time_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"time": anime.Broadcast.Time,
},
nil,
)
return time.Time{}
}
localAfter := after.In(loc)
daysAhead := (int(day) - int(localAfter.Weekday()) + 7) % 7
candidate := time.Date(localAfter.Year(), localAfter.Month(), localAfter.Day()+daysAhead, hour, minute, 0, 0, loc)
if !candidate.After(localAfter) {
candidate = candidate.AddDate(0, 0, 7)
}
return candidate.UTC()
}
func weekdayFromJikan(day string) time.Weekday {
switch strings.ToLower(strings.TrimSpace(day)) {
case "sundays":
return time.Sunday
case "mondays":
return time.Monday
case "tuesdays":
return time.Tuesday
case "wednesdays":
return time.Wednesday
case "thursdays":
return time.Thursday
case "fridays":
return time.Friday
case "saturdays":
return time.Saturday
default:
return -1
}
}
func parseBroadcastTime(value string) (int, int, bool) {
t, err := time.Parse("15:04", strings.TrimSpace(value))
if err != nil {
return 0, 0, false
}
return t.Hour(), t.Minute(), true
}
func truncate(value string, maxLen int) string {
if len(value) <= maxLen {
return value
}
return value[:maxLen]
}

60
internal/observability/fx.go Normal file
View File

@@ -0,0 +1,60 @@
package observability
import (
"go.uber.org/fx/fxevent"
)
type fxLogger struct{}
func NewFxLogger() fxevent.Logger {
return fxLogger{}
}
func (fxLogger) LogEvent(event fxevent.Event) {
eventName, fields, err := describeFXEventError(event)
if err == nil {
return
}
Error(eventName, "fx", "", fields, err)
}
func describeFXEventError(event fxevent.Event) (string, map[string]any, error) {
if ok, eventName, fields, err := describeFXExecutionEventError(event); ok {
return eventName, fields, err
}
return describeFXLifecycleEventError(event)
}
func describeFXExecutionEventError(event fxevent.Event) (bool, string, map[string]any, error) {
switch e := event.(type) {
case *fxevent.Provided:
return true, "fx_provide_failed", map[string]any{"constructor": e.ConstructorName}, e.Err
case *fxevent.Invoked:
return true, "fx_invoke_failed", map[string]any{"function": e.FunctionName}, e.Err
case *fxevent.Run:
return true, "fx_run_failed", map[string]any{"function": e.Name, "kind": e.Kind}, e.Err
case *fxevent.OnStartExecuted:
return true, "fx_on_start_failed", map[string]any{"caller": e.CallerName, "function": e.FunctionName, "runtime": e.Runtime}, e.Err
case *fxevent.OnStopExecuted:
return true, "fx_on_stop_failed", map[string]any{"caller": e.CallerName, "function": e.FunctionName, "runtime": e.Runtime}, e.Err
default:
return false, "", nil, nil
}
}
func describeFXLifecycleEventError(event fxevent.Event) (string, map[string]any, error) {
switch e := event.(type) {
case *fxevent.Started:
return "fx_start_failed", nil, e.Err
case *fxevent.Stopped:
return "fx_stop_failed", nil, e.Err
case *fxevent.RollingBack:
return "fx_rollback_start", nil, e.StartErr
case *fxevent.RolledBack:
return "fx_rollback_failed", nil, e.Err
default:
return "", nil, nil
}
}

10
internal/observability/helpers.go
View File

@@ -1,5 +1,7 @@
package observability
import "context"
// Small helpers to keep logging consistent and low-friction across the codebase.
func Info(event string, component string, message string, fields map[string]any) {
@@ -10,6 +12,14 @@ func Warn(event string, component string, message string, fields map[string]any,
LogJSON(LogLevelWarn, event, component, message, fields, err)
}
func WarnContext(ctx context.Context, event string, component string, message string, fields map[string]any, err error) {
LogContext(ctx, LogLevelWarn, event, component, message, fields, err)
}
func Error(event string, component string, message string, fields map[string]any, err error) {
LogJSON(LogLevelError, event, component, message, fields, err)
}
func ErrorContext(ctx context.Context, event string, component string, message string, fields map[string]any, err error) {
LogContext(ctx, LogLevelError, event, component, message, fields, err)
}

35
internal/observability/helpers_test.go Normal file
View File

@@ -0,0 +1,35 @@
package observability
import (
"errors"
"fmt"
"strings"
"testing"
)
func TestWarnEnrichesSourceAndErrorContext(t *testing.T) {
fields := enrichFields(LogLevelWarn, map[string]any{"anime_id": 123}, wrappedError())
if fields["anime_id"] != 123 {
t.Fatalf("expected existing field to survive, got %#v", fields["anime_id"])
}
source, ok := fields["source"].(string)
if !ok || source == "" {
t.Fatalf("expected source field, got %#v", fields["source"])
}
errorType, ok := fields["error_type"].(string)
if !ok || errorType == "" {
t.Fatalf("expected error_type field, got %#v", fields["error_type"])
}
chain, ok := fields["error_chain"].(string)
if !ok || !strings.Contains(chain, "query anime") || !strings.Contains(chain, "db timeout") {
t.Fatalf("expected wrapped error chain, got %#v", fields["error_chain"])
}
}
func wrappedError() error {
return fmt.Errorf("query anime: %w", errors.New("db timeout"))
}

451
internal/observability/log.go
View File

@@ -2,11 +2,33 @@
package observability
import (
"encoding/json"
"context"
"errors"
"fmt"
"log"
"net"
"os"
"path/filepath"
"reflect"
"runtime"
"sort"
"strconv"
"strings"
"time"
)
const (
ansiReset = "\x1b[0m"
ansiBlue = "\x1b[36m"
ansiStatusBlue = "\x1b[34m"
ansiGreen = "\x1b[32m"
ansiYellow = "\x1b[33m"
ansiOrange = "\x1b[38;5;208m"
ansiRed = "\x1b[31m"
)
var colorLogs = shouldColorLogs()
type LogLevel string
const (
@@ -25,35 +47,428 @@ type LogEvent struct {
Component string `json:"component,omitempty"`
}
func LogJSON(level LogLevel, event string, component string, message string, fields map[string]any, err error) {
errorValue := ""
if err != nil {
errorValue = err.Error()
func init() {
log.SetFlags(0)
}
func LogJSON(level LogLevel, event string, component string, message string, fields map[string]any, err error) {
LogContext(context.TODO(), level, event, component, message, fields, err)
}
func LogContext(ctx context.Context, level LogLevel, event string, component string, message string, fields map[string]any, err error) {
fields = enrichFields(level, fields, err)
fields = enrichRequestFields(ctx, fields)
entry := LogEvent{
TS: time.Now().UTC().Format(time.RFC3339Nano),
Level: level,
Event: event,
Message: message,
Fields: fields,
Error: errorValue,
Component: component,
}
// Best-effort. If encoding fails, fall back to a minimal line.
bytes, marshalErr := json.Marshal(entry)
if marshalErr != nil {
// Keep output JSON-only even on failures by constructing a minimal entry.
// Marshal individual strings to ensure proper escaping.
tsBytes, _ := json.Marshal(time.Now().UTC().Format(time.RFC3339Nano))
levelBytes, _ := json.Marshal(level)
eventBytes, _ := json.Marshal("log_marshal_failed")
componentBytes, _ := json.Marshal(component)
errBytes, _ := json.Marshal(marshalErr.Error())
log.Printf(`{"ts":%s,"level":%s,"event":%s,"component":%s,"error":%s}`, tsBytes, levelBytes, eventBytes, componentBytes, errBytes)
if err != nil {
entry.Error = err.Error()
}
log.Print(formatLogEntry(entry))
}
func enrichRequestFields(ctx context.Context, fields map[string]any) map[string]any {
requestContext, ok := RequestContextFromContext(ctx)
if !ok {
return fields
}
enriched := cloneFields(fields)
if enriched == nil {
enriched = make(map[string]any, 3)
}
if requestContext.ID != "" {
if _, exists := enriched["request_id"]; !exists {
enriched["request_id"] = requestContext.ID
}
}
if requestContext.Path != "" {
if _, exists := enriched["request_path"]; !exists {
enriched["request_path"] = requestContext.Path
}
}
if requestContext.Route != "" && requestContext.Route != requestContext.Path {
if _, exists := enriched["request_route"]; !exists {
enriched["request_route"] = requestContext.Route
}
}
return enriched
}
func enrichFields(level LogLevel, fields map[string]any, err error) map[string]any {
if level == LogLevelInfo {
return fields
}
enriched := cloneFields(fields)
if enriched == nil {
enriched = make(map[string]any, 3)
}
if _, exists := enriched["source"]; !exists {
if source := callerSource(); source != "" {
enriched["source"] = source
}
}
if err != nil {
if _, exists := enriched["error_type"]; !exists {
if errorType := formatErrorType(err); errorType != "" {
enriched["error_type"] = errorType
}
}
if _, exists := enriched["error_chain"]; !exists {
if chain := formatErrorChain(err); chain != "" {
enriched["error_chain"] = chain
}
}
}
return enriched
}
func callerSource() string {
pcs := make([]uintptr, 8)
n := runtime.Callers(3, pcs)
frames := runtime.CallersFrames(pcs[:n])
for {
frame, more := frames.Next()
if !strings.Contains(frame.File, "/internal/observability/") {
return filepath.Base(frame.File) + ":" + strconv.Itoa(frame.Line)
}
if !more {
return ""
}
}
}
func formatErrorType(err error) string {
errType := reflect.TypeOf(err)
if errType == nil {
return ""
}
return errType.String()
}
func formatErrorChain(err error) string {
parts := make([]string, 0, 4)
for current := err; current != nil; current = errors.Unwrap(current) {
parts = append(parts, current.Error())
if len(parts) == 4 {
break
}
}
if len(parts) <= 1 {
return ""
}
return strings.Join(parts, " -> ")
}
func formatLogEntry(entry LogEvent) string {
if entry.Event == "http_request" {
return formatHTTPRequestLog(entry)
}
parts := []string{entry.TS, formatLogLevel(entry.Level), entry.Event}
if entry.Component != "" {
parts = append(parts, "component="+entry.Component)
}
if entry.Message != "" {
parts = append(parts, quoteIfNeeded(entry.Message))
}
if len(entry.Fields) > 0 {
keys := make([]string, 0, len(entry.Fields))
for key := range entry.Fields {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
parts = append(parts, key+"="+formatFieldValue(entry.Fields[key]))
}
}
if entry.Error != "" {
parts = append(parts, "error="+quoteIfNeeded(entry.Error))
}
return strings.Join(parts, " ")
}
func formatHTTPRequestLog(entry LogEvent) string {
fields := cloneFields(entry.Fields)
status := popField(fields, "status")
method := popField(fields, "method")
path := popField(fields, "path")
duration := popField(fields, "duration_ms")
bytes := popField(fields, "bytes")
route := popField(fields, "route")
query := popField(fields, "query")
clientIP := popField(fields, "client_ip")
parts := []string{entry.TS, formatLogLevel(entry.Level), "http"}
appendNonEmpty(&parts, status)
appendNonEmpty(&parts, strings.TrimSpace(method+" "+path))
appendNonEmpty(&parts, duration)
appendNonEmpty(&parts, bytes)
appendKeyValue(&parts, "route", route)
appendKeyValueQuoted(&parts, "query", query)
appendClientIP(&parts, clientIP)
appendSortedFields(&parts, fields)
if entry.Error != "" {
parts = append(parts, "error="+quoteIfNeeded(entry.Error))
}
return strings.Join(parts, " ")
}
func appendNonEmpty(parts *[]string, value string) {
if value == "" {
return
}
log.Print(string(bytes))
*parts = append(*parts, value)
}
func appendKeyValue(parts *[]string, key string, value string) {
if value == "" {
return
}
*parts = append(*parts, key+"="+value)
}
func appendKeyValueQuoted(parts *[]string, key string, value string) {
if value == "" {
return
}
*parts = append(*parts, key+"="+quoteIfNeeded(value))
}
func appendClientIP(parts *[]string, clientIP string) {
if clientIP == "" || isLocalClientIP(clientIP) {
return
}
*parts = append(*parts, "ip="+clientIP)
}
func appendSortedFields(parts *[]string, fields map[string]any) {
if len(fields) == 0 {
return
}
keys := make([]string, 0, len(fields))
for key := range fields {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
*parts = append(*parts, key+"="+formatFieldValue(fields[key]))
}
}
func cloneFields(fields map[string]any) map[string]any {
if len(fields) == 0 {
return nil
}
copyFields := make(map[string]any, len(fields))
for key, value := range fields {
copyFields[key] = value
}
return copyFields
}
func popField(fields map[string]any, key string) string {
if len(fields) == 0 {
return ""
}
value, ok := fields[key]
if !ok {
return ""
}
delete(fields, key)
return formatInlineField(key, value)
}
func formatInlineField(key string, value any) string {
switch key {
case "status":
return formatHTTPStatus(value)
case "duration_ms":
return formatDurationMillis(value)
case "bytes":
return formatBytes(value)
default:
if text, ok := value.(string); ok {
return text
}
return fmt.Sprint(value)
}
}
func formatHTTPStatus(value any) string {
status := fmt.Sprint(value)
if !colorLogs || status == "" {
return status
}
switch status[0] {
case '1':
return ansiStatusBlue + status + ansiReset
case '2':
return ansiGreen + status + ansiReset
case '3':
return ansiYellow + status + ansiReset
case '4':
return ansiOrange + status + ansiReset
case '5':
return ansiRed + status + ansiReset
default:
return status
}
}
func formatDurationMillis(value any) string {
ms, ok := toFloat64(value)
if !ok {
return fmt.Sprint(value)
}
return strconv.FormatFloat(ms, 'f', -1, 64) + "ms"
}
func formatBytes(value any) string {
bytesValue, ok := toFloat64(value)
if !ok {
return fmt.Sprint(value)
}
if bytesValue < 1024 {
return strconv.FormatFloat(bytesValue, 'f', -1, 64) + "B"
}
if bytesValue < 1024*1024 {
return strconv.FormatFloat(bytesValue/1024, 'f', 1, 64) + "KB"
}
return strconv.FormatFloat(bytesValue/(1024*1024), 'f', 1, 64) + "MB"
}
func toFloat64(value any) (float64, bool) {
switch v := value.(type) {
case int:
return float64(v), true
case int32:
return float64(v), true
case int64:
return float64(v), true
case float32:
return float64(v), true
case float64:
return v, true
default:
return 0, false
}
}
func isLocalClientIP(value string) bool {
parsed := net.ParseIP(value)
if parsed == nil {
return false
}
return parsed.IsLoopback()
}
func formatLogLevel(level LogLevel) string {
if colorLogs {
switch level {
case LogLevelWarn:
return ansiYellow + "WARN" + ansiReset
case LogLevelError:
return ansiRed + "ERROR" + ansiReset
default:
return ansiBlue + "INFO" + ansiReset
}
}
switch level {
case LogLevelWarn:
return "WARN"
case LogLevelError:
return "ERROR"
default:
return "INFO"
}
}
func shouldColorLogs() bool {
if strings.TrimSpace(os.Getenv("NO_COLOR")) != "" {
return false
}
if strings.EqualFold(strings.TrimSpace(os.Getenv("TERM")), "dumb") {
return false
}
info, err := os.Stderr.Stat()
if err != nil {
return false
}
return info.Mode()&os.ModeCharDevice != 0
}
func formatFieldValue(value any) string {
switch v := value.(type) {
case string:
return quoteIfNeeded(v)
case time.Duration:
return v.String()
case float32:
return strconv.FormatFloat(float64(v), 'f', -1, 32)
case float64:
return strconv.FormatFloat(v, 'f', -1, 64)
case fmt.Stringer:
return quoteIfNeeded(v.String())
default:
return quoteIfNeeded(fmt.Sprint(value))
}
}
func quoteIfNeeded(value string) string {
if value == "" {
return `""`
}
for _, r := range value {
if r == '=' || r == ' ' || r == '\t' || r == '\n' || r == '"' {
return strconv.Quote(value)
}
}
return value
}

60
internal/observability/log_test.go Normal file
View File

@@ -0,0 +1,60 @@
package observability
import (
"strings"
"testing"
)
func TestFormatLogEntryFormatsHTTPRequestCompactly(t *testing.T) {
line := formatLogEntry(LogEvent{
TS: "2026-06-11T12:57:39.557972Z",
Level: LogLevelInfo,
Event: "http_request",
Fields: map[string]any{
"bytes": 56198,
"client_ip": "127.0.0.1",
"duration_ms": 9.419,
"method": "GET",
"path": "/api/catalog/top-pick",
"status": 200,
},
})
checks := []string{
"2026-06-11T12:57:39.557972Z INFO http 200 GET /api/catalog/top-pick 9.419ms 54.9KB",
}
for _, check := range checks {
if !strings.Contains(line, check) {
t.Fatalf("line %q missing %q", line, check)
}
}
if strings.Contains(line, "client_ip=") {
t.Fatalf("line should omit loopback ip: %q", line)
}
}
func TestFormatHTTPStatusColorsByStatusFamily(t *testing.T) {
previousColorLogs := colorLogs
colorLogs = true
t.Cleanup(func() {
colorLogs = previousColorLogs
})
tests := map[any]string{
101: ansiStatusBlue + "101" + ansiReset,
200: ansiGreen + "200" + ansiReset,
302: ansiYellow + "302" + ansiReset,
404: ansiOrange + "404" + ansiReset,
500: ansiRed + "500" + ansiReset,
"unknown": "unknown",
}
for input, want := range tests {
got := formatHTTPStatus(input)
if got != want {
t.Fatalf("formatHTTPStatus(%v) = %q, want %q", input, got, want)
}
}
}

3
internal/observability/metrics_test.go
View File

@@ -1,6 +1,7 @@
package observability
import (
"context"
"io"
"net/http"
"net/http/httptest"
@@ -17,7 +18,7 @@ func TestMetricsHandlerRendersPrometheusFamilies(t *testing.T) {
metrics.ObserveCache("jikan", "hit")
metrics.ObserveCache("episode_availability", "miss")
req := httptest.NewRequest(http.MethodGet, "/metrics", nil)
req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/metrics", nil)
rec := httptest.NewRecorder()
metrics.Handler().ServeHTTP(rec, req)

32
internal/observability/request.go Normal file
View File

@@ -0,0 +1,32 @@
package observability
import "context"
type requestContextKey struct{}
type RequestContext struct {
ID string
Path string
Route string
}
func WithRequestContext(ctx context.Context, requestID string, path string, route string) context.Context {
if ctx == nil {
return nil
}
return context.WithValue(ctx, requestContextKey{}, RequestContext{
ID: requestID,
Path: path,
Route: route,
})
}
func RequestContextFromContext(ctx context.Context) (RequestContext, bool) {
if ctx == nil {
return RequestContext{}, false
}
requestContext, ok := ctx.Value(requestContextKey{}).(RequestContext)
return requestContext, ok
}

21
internal/playback/handler/handler.go
View File

@@ -3,9 +3,11 @@ package handler
import (
"context"
"errors"
"fmt"
"io"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
netutil "mal/pkg/net"
"net/http"
@@ -37,7 +39,6 @@ func NewPlaybackHandler(svc domain.PlaybackService, animeSvc domain.AnimePlaybac
}
func (h *PlaybackHandler) Register(r *gin.Engine) {
r.GET("/anime/:id/watch", h.HandleWatchPage)
r.POST("/api/watch-progress", h.HandleSaveProgress)
r.POST("/api/watch-complete", h.HandleWatchComplete)
@@ -302,6 +303,10 @@ func (h *PlaybackHandler) HandleProxyStream(c *gin.Context) {
resp, err := h.streamingClient.Do(req)
if err != nil {
if !errors.Is(err, context.Canceled) {
observability.ErrorContext(c.Request.Context(), "proxy_stream_upstream_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
}
c.Status(http.StatusBadGateway)
return
}
@@ -310,11 +315,15 @@ func (h *PlaybackHandler) HandleProxyStream(c *gin.Context) {
if isHLSPlaylistResponse(targetURL, resp.Header) {
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_stream_playlist_read_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway)
return
}
rewritten, err := h.rewriteHLSPlaylist(string(body), targetURL, referer)
if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_stream_playlist_rewrite_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway)
return
}
@@ -415,7 +424,9 @@ func (h *PlaybackHandler) proxyPlaylistURI(rawURI string, baseURL *url.URL, refe
if err != nil {
return "", err
}
return "/watch/proxy/stream?token=" + url.QueryEscape(token), nil
params := url.Values{}
params.Set("token", token)
return "/watch/proxy/stream?" + params.Encode(), nil
}
func copyProxyHeaders(dst http.Header, src http.Header) {
@@ -482,6 +493,10 @@ func (h *PlaybackHandler) HandleProxySubtitle(c *gin.Context) {
resp, err := h.proxyClient.Do(req)
if err != nil {
if !errors.Is(err, context.Canceled) {
observability.ErrorContext(c.Request.Context(), "proxy_subtitle_upstream_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
}
c.Status(http.StatusBadGateway)
return
}
@@ -489,6 +504,8 @@ func (h *PlaybackHandler) HandleProxySubtitle(c *gin.Context) {
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_subtitle_read_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway)
return
}

11
internal/playback/module.go
View File

@@ -1,7 +1,6 @@
package playback
import (
"mal/integrations/jikan"
"mal/integrations/playback/allanime"
"mal/internal/config"
"mal/internal/domain"
@@ -18,14 +17,8 @@ func provideProxyTokenKey(cfg config.Config) ProxyTokenKey {
var Module = fx.Options(
fx.Provide(
NewPlaybackRepository,
fx.Annotate(
func(repo domain.PlaybackRepository, providers []domain.Provider, jikan *jikan.Client, episodeSvc domain.EpisodeService, auditSvc domain.AuditService, proxyTokenKey ProxyTokenKey) domain.PlaybackService {
return NewPlaybackService(repo, providers, jikan, episodeSvc, auditSvc, proxyTokenKey)
},
),
func(svc domain.PlaybackService, animeSvc domain.AnimePlaybackService) *handler.PlaybackHandler {
return handler.NewPlaybackHandler(svc, animeSvc)
},
NewPlaybackService,
handler.NewPlaybackHandler,
),
fx.Provide(
server.AsRouteRegister(func(h *handler.PlaybackHandler) server.RouteRegister {

193
internal/playback/progress.go Normal file
View File

@@ -0,0 +1,193 @@
package playback
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"strconv"
"github.com/google/uuid"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *playbackService) loadWatchProgress(ctx context.Context, userID string, animeID int, totalEpisodes int, episode string) (float64, string, []int64) {
if userID == "" {
return 0, "", nil
}
entry, err := s.repo.GetWatchListEntry(ctx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
watchlistStatus := ""
var watchlistIDs []int64
startTime := 0.0
if err == nil {
watchlistStatus = entry.Status
watchlistIDs = []int64{entry.AnimeID}
if resumeTimeForEpisode(entry.CurrentEpisode, entry.CurrentTimeSeconds, totalEpisodes, episode) > 0 {
startTime = entry.CurrentTimeSeconds
}
}
if startTime > 0 {
return startTime, watchlistStatus, watchlistIDs
}
cwEntry, err := s.repo.GetContinueWatchingEntry(ctx, db.GetContinueWatchingEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
startTime = resumeTimeForEpisode(cwEntry.CurrentEpisode, cwEntry.CurrentTimeSeconds, totalEpisodes, episode)
}
return startTime, watchlistStatus, watchlistIDs
}
func resumeTimeForEpisode(currentEpisode sql.NullInt64, currentTimeSeconds float64, totalEpisodes int, requestedEpisode string) float64 {
if !currentEpisode.Valid {
return 0
}
if strconv.FormatInt(currentEpisode.Int64, 10) == requestedEpisode {
return currentTimeSeconds
}
if totalEpisodes > 0 && requestedEpisode == strconv.Itoa(totalEpisodes) && currentEpisode.Int64 == int64(totalEpisodes) {
return currentTimeSeconds
}
return 0
}
func (s *playbackService) CompleteAnime(ctx context.Context, userID string, animeID int64) error {
if err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
entry, err := repo.GetWatchListEntry(txCtx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: animeID,
})
if err != nil || entry.Status != "completed" {
_, err = repo.UpsertWatchListEntry(txCtx, db.UpsertWatchListEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Status: "completed",
CurrentEpisode: entry.CurrentEpisode,
CurrentTimeSeconds: entry.CurrentTimeSeconds,
})
if err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
if err := s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_completed",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
}); err != nil {
observability.Warn(
"audit_record_failed",
"playback",
"",
map[string]any{"user_id": userID, "anime_id": animeID, "action": "watch_completed"},
err,
)
}
return nil
}
func (s *playbackService) SaveProgress(ctx context.Context, userID string, animeID int64, episode int, timeSeconds float64) error {
err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
if _, err := repo.GetAnime(txCtx, animeID); err != nil {
if _, err := repo.UpsertAnime(txCtx, minimalAnimeParams(animeID)); err != nil {
return err
}
}
_, err := repo.UpsertContinueWatchingEntry(txCtx, db.UpsertContinueWatchingEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
CurrentEpisode: sql.NullInt64{Int64: int64(episode), Valid: true},
CurrentTimeSeconds: timeSeconds,
DurationSeconds: sql.NullFloat64{Valid: false},
})
return err
})
if err != nil {
return err
}
metadataBytes, marshalErr := json.Marshal(struct {
Episode int `json:"episode"`
TimeSeconds float64 `json:"time_seconds"`
}{Episode: episode, TimeSeconds: timeSeconds})
if marshalErr == nil {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
MetadataJSON: metadataBytes,
})
} else {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
})
}
observability.Info("watch_progress_saved", "playback", "", map[string]any{
"anime_id": animeID,
"episode": episode,
"time_seconds": timeSeconds,
"user_id": userID,
})
return nil
}
func (s *playbackService) ensureAnimeRow(ctx context.Context, anime domain.Anime) {
if _, err := s.repo.GetAnime(ctx, int64(anime.MalID)); err == nil {
return
}
_, _ = s.repo.UpsertAnime(ctx, animeParams(anime))
}
func animeParams(anime domain.Anime) db.UpsertAnimeParams {
durationSeconds := anime.DurationSeconds()
duration := sql.NullFloat64{Valid: durationSeconds > 0}
if duration.Valid {
duration.Float64 = durationSeconds
}
return db.UpsertAnimeParams{
ID: int64(anime.MalID),
TitleOriginal: anime.Title,
TitleEnglish: sql.NullString{String: anime.TitleEnglish, Valid: anime.TitleEnglish != ""},
TitleJapanese: sql.NullString{String: anime.TitleJapanese, Valid: anime.TitleJapanese != ""},
ImageUrl: anime.ImageURL(),
Airing: sql.NullBool{Bool: anime.Airing, Valid: true},
DurationSeconds: duration,
}
}
func minimalAnimeParams(animeID int64) db.UpsertAnimeParams {
return db.UpsertAnimeParams{
ID: animeID,
TitleOriginal: fmt.Sprintf("Anime %d", animeID),
Airing: sql.NullBool{Valid: false},
}
}

69
internal/playback/proxy_tokens.go Normal file
View File

@@ -0,0 +1,69 @@
package playback
import (
"crypto/rand"
"encoding/base64"
"fmt"
"sync"
"time"
)
type proxyTokenTarget struct {
targetURL string
referer string
scope string
expiresAt time.Time
}
type proxyTokenStore struct {
mu sync.Mutex
tokens map[string]proxyTokenTarget
}
func newProxyTokenStore() *proxyTokenStore {
return &proxyTokenStore{
tokens: make(map[string]proxyTokenTarget),
}
}
func (s *proxyTokenStore) create(targetURL, referer, scope string, ttl time.Duration, now time.Time) (string, error) {
tokenBytes := make([]byte, 32)
if _, err := rand.Read(tokenBytes); err != nil {
return "", fmt.Errorf("generate proxy token: %w", err)
}
token := base64.RawURLEncoding.EncodeToString(tokenBytes)
s.mu.Lock()
defer s.mu.Unlock()
s.pruneExpiredLocked(now)
s.tokens[token] = proxyTokenTarget{
targetURL: targetURL,
referer: referer,
scope: scope,
expiresAt: now.Add(ttl),
}
return token, nil
}
func (s *proxyTokenStore) resolve(token string, now time.Time) (proxyTokenTarget, error) {
s.mu.Lock()
defer s.mu.Unlock()
target, ok := s.tokens[token]
if !ok {
return proxyTokenTarget{}, fmt.Errorf("invalid proxy token")
}
if !target.expiresAt.After(now) {
delete(s.tokens, token)
return proxyTokenTarget{}, fmt.Errorf("proxy token expired")
}
return target, nil
}
func (s *proxyTokenStore) pruneExpiredLocked(now time.Time) {
for token, target := range s.tokens {
if !target.expiresAt.After(now) {
delete(s.tokens, token)
}
}
}

8
internal/playback/repository.go
View File

@@ -23,6 +23,14 @@ func (r *playbackRepository) InTx(ctx context.Context, fn func(ctx context.Conte
}, fn)
}
func (r *playbackRepository) UpsertAnime(ctx context.Context, params db.UpsertAnimeParams) (db.Anime, error) {
return r.queries.UpsertAnime(ctx, params)
}
func (r *playbackRepository) GetAnime(ctx context.Context, id int64) (db.Anime, error) {
return r.queries.GetAnime(ctx, id)
}
func (r *playbackRepository) GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error) {
return r.queries.GetWatchListEntry(ctx, params)
}

483
internal/playback/service.go
View File

@@ -3,26 +3,12 @@ package playback
import (
"context"
"crypto/rand"
"database/sql"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
netutil "mal/pkg/net"
"net/http"
"net/url"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/google/uuid"
)
type playbackService struct {
@@ -38,66 +24,6 @@ type playbackService struct {
type ProxyTokenKey string
type proxyTokenTarget struct {
targetURL string
referer string
scope string
expiresAt time.Time
}
type proxyTokenStore struct {
mu sync.Mutex
tokens map[string]proxyTokenTarget
}
func newProxyTokenStore() *proxyTokenStore {
return &proxyTokenStore{
tokens: make(map[string]proxyTokenTarget),
}
}
func (s *proxyTokenStore) create(targetURL, referer, scope string, ttl time.Duration, now time.Time) (string, error) {
tokenBytes := make([]byte, 32)
if _, err := rand.Read(tokenBytes); err != nil {
return "", fmt.Errorf("generate proxy token: %w", err)
}
token := base64.RawURLEncoding.EncodeToString(tokenBytes)
s.mu.Lock()
defer s.mu.Unlock()
s.pruneExpiredLocked(now)
s.tokens[token] = proxyTokenTarget{
targetURL: targetURL,
referer: referer,
scope: scope,
expiresAt: now.Add(ttl),
}
return token, nil
}
func (s *proxyTokenStore) resolve(token string, now time.Time) (proxyTokenTarget, error) {
s.mu.Lock()
defer s.mu.Unlock()
target, ok := s.tokens[token]
if !ok {
return proxyTokenTarget{}, fmt.Errorf("invalid proxy token")
}
if !target.expiresAt.After(now) {
delete(s.tokens, token)
return proxyTokenTarget{}, fmt.Errorf("proxy token expired")
}
return target, nil
}
func (s *proxyTokenStore) pruneExpiredLocked(now time.Time) {
for token, target := range s.tokens {
if !target.expiresAt.After(now) {
delete(s.tokens, token)
}
}
}
func NewPlaybackService(repo domain.PlaybackRepository, providers []domain.Provider, jikan *jikan.Client, episodes domain.EpisodeService, auditSvc domain.AuditService, proxyTokenKey ProxyTokenKey) domain.PlaybackService {
return &playbackService{
repo: repo,
@@ -132,408 +58,11 @@ func (s *playbackService) ResolveProxyToken(token string, scope string) (string,
return target.targetURL, target.referer, nil
}
func (s *playbackService) BuildWatchData(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string, userID string) (domain.WatchPageData, error) {
// 1. Get Anime details for total episodes and titles
anime, err := s.jikan.GetAnimeByID(ctx, animeID)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch anime: %w", err)
}
// 2. Resolve streams from providers
searchTitles := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
searchTitles = append(searchTitles, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
searchTitles = append(searchTitles, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
searchTitles = append(searchTitles, syn)
}
}
canonicalEpisodes, err := s.episodes.GetCanonicalEpisodes(ctx, domain.Anime{Anime: anime}, false)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch episodes: %w", err)
}
requestedMode := mode
modeSwitchedFrom := ""
if epNum, parseErr := strconv.Atoi(episode); parseErr == nil && requestedMode == "dub" {
for _, ep := range canonicalEpisodes.Episodes {
if ep.Number == epNum && !ep.HasDub && ep.HasSub {
mode = "sub"
modeSwitchedFrom = requestedMode
break
}
}
}
modeSources := map[string]domain.ModeSource{}
var result *domain.StreamResult
for _, m := range []string{"sub", "dub"} {
for _, p := range s.providers {
res, err := p.GetStreams(ctx, animeID, searchTitles, episode, m)
if err != nil || res == nil {
continue
}
var subItems []domain.SubtitleItem
for _, sub := range res.Subtitles {
subToken, _ := s.SignProxyToken(sub.URL, res.Referer, "subtitle")
subItems = append(subItems, domain.SubtitleItem{
Lang: sub.Label,
Token: subToken,
})
}
streamToken, _ := s.SignProxyToken(res.URL, res.Referer, "stream")
modeSources[m] = domain.ModeSource{
Token: streamToken,
Subtitles: subItems,
}
if m == mode {
result = res
}
break
}
}
if len(modeSources) == 0 {
return domain.WatchPageData{}, fmt.Errorf("no streams found")
}
if result == nil {
return domain.WatchPageData{}, fmt.Errorf("no streams found for mode %s", mode)
}
// 3. Get start time from progress
startTime := 0.0
var watchlistStatus string
var watchlistIDs []int64
if userID != "" {
entry, err := s.repo.GetWatchListEntry(ctx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
watchlistStatus = entry.Status
watchlistIDs = []int64{entry.AnimeID}
if entry.CurrentEpisode.Valid && strconv.FormatInt(entry.CurrentEpisode.Int64, 10) == episode {
startTime = entry.CurrentTimeSeconds
} else if anime.Episodes > 0 && episode == strconv.Itoa(anime.Episodes) && entry.CurrentEpisode.Valid && entry.CurrentEpisode.Int64 == int64(anime.Episodes) {
startTime = entry.CurrentTimeSeconds
}
}
// Fall back to continue_watching_entry for progress if not in watchlist
if startTime == 0 {
cwEntry, err := s.repo.GetContinueWatchingEntry(ctx, db.GetContinueWatchingEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
if cwEntry.CurrentEpisode.Valid && strconv.FormatInt(cwEntry.CurrentEpisode.Int64, 10) == episode {
startTime = cwEntry.CurrentTimeSeconds
} else if anime.Episodes > 0 && episode == strconv.Itoa(anime.Episodes) && cwEntry.CurrentEpisode.Valid && cwEntry.CurrentEpisode.Int64 == int64(anime.Episodes) {
startTime = cwEntry.CurrentTimeSeconds
}
}
}
}
// 5. Build provider data
streams := []domain.ProviderStream{
{
Name: "Primary",
Quality: "Auto",
MalID: animeID,
IsCurrent: true,
},
}
go s.warmStreamURL(result.URL, result.Referer)
// 6. Resolve relations/seasons
relations, _ := s.jikan.GetFullRelations(ctx, animeID)
var seasons []domain.SeasonEntry
tvCounter := 1
for _, rel := range relations {
if strings.ToLower(rel.Anime.Type) == "tv" || strings.ToLower(rel.Anime.Type) == "movie" {
seasons = append(seasons, domain.SeasonEntry{
MalID: rel.Anime.MalID,
Title: rel.Anime.DisplayTitle(),
Prefix: rel.Relation,
IsCurrent: rel.IsCurrent,
})
if rel.Relation == "TV" {
seasons[len(seasons)-1].Prefix = fmt.Sprintf("S%d", tvCounter)
tvCounter++
}
}
}
// Final assembly
segments := s.fetchSkipSegments(ctx, userID, animeID, episode)
watchData := domain.WatchData{
MalID: animeID,
Title: anime.DisplayTitle(),
CurrentEpisode: episode,
StartTimeSeconds: startTime,
Episodes: canonicalEpisodes.Episodes,
Providers: []domain.ProviderData{
{Streams: streams},
},
ModeSources: modeSources,
InitialMode: mode,
ModeSwitchedFrom: modeSwitchedFrom,
AvailableModes: func() []string {
var modes []string
for m := range modeSources {
modes = append(modes, m)
}
sort.Strings(modes)
return modes
}(),
Segments: segments,
Airing: anime.Airing,
}
return domain.WatchPageData{
WatchData: watchData,
Anime: domain.Anime{Anime: anime},
Episodes: canonicalEpisodes.Episodes,
CurrentEpID: episode,
WatchlistStatus: watchlistStatus,
WatchlistIDs: watchlistIDs,
Seasons: seasons,
}, nil
}
func (s *playbackService) CompleteAnime(ctx context.Context, userID string, animeID int64) error {
if err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
entry, err := repo.GetWatchListEntry(txCtx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: animeID,
})
if err != nil || entry.Status != "completed" {
_, err = repo.UpsertWatchListEntry(txCtx, db.UpsertWatchListEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Status: "completed",
CurrentEpisode: entry.CurrentEpisode,
CurrentTimeSeconds: entry.CurrentTimeSeconds,
})
if err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
if err := s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_completed",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
}); err != nil {
observability.Warn(
"audit_record_failed",
"playback",
"",
map[string]any{"user_id": userID, "anime_id": animeID, "action": "watch_completed"},
err,
)
}
return nil
}
func (s *playbackService) SaveProgress(ctx context.Context, userID string, animeID int64, episode int, timeSeconds float64) error {
_, err := s.repo.UpsertContinueWatchingEntry(ctx, db.UpsertContinueWatchingEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
CurrentEpisode: sql.NullInt64{Int64: int64(episode), Valid: true},
CurrentTimeSeconds: timeSeconds,
DurationSeconds: sql.NullFloat64{Valid: false},
})
if err != nil {
return err
}
metadataBytes, marshalErr := json.Marshal(struct {
Episode int `json:"episode"`
TimeSeconds float64 `json:"time_seconds"`
}{Episode: episode, TimeSeconds: timeSeconds})
if marshalErr == nil {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
MetadataJSON: metadataBytes,
})
} else {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
})
}
observability.Info("watch_progress_saved", "playback", "", map[string]any{
"anime_id": animeID,
"episode": episode,
"time_seconds": timeSeconds,
"user_id": userID,
})
return nil
}
func (s *playbackService) UpsertSkipSegmentOverride(ctx context.Context, userID string, animeID int64, episode int, skipType string, startTime, endTime float64) error {
if userID == "" {
return fmt.Errorf("not authenticated")
}
if animeID <= 0 || episode <= 0 {
return fmt.Errorf("invalid anime/episode")
}
t := strings.ToLower(strings.TrimSpace(skipType))
switch t {
case "op", "opening", "intro":
t = "op"
case "ed", "ending", "outro":
t = "ed"
default:
return fmt.Errorf("invalid skip_type")
}
if !(startTime >= 0) || !(endTime > startTime) {
return fmt.Errorf("invalid interval")
}
// let the player-side filters ignore obviously wrong durations, but keep some sanity.
if endTime-startTime < 5 || endTime-startTime > 10*60 {
return fmt.Errorf("interval duration out of range")
}
return s.repo.UpsertSkipSegmentOverride(ctx, db.SkipSegmentOverrideRow{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Episode: int64(episode),
SkipType: t,
StartTime: startTime,
EndTime: endTime,
})
}
func (s *playbackService) fetchSkipSegments(ctx context.Context, userID string, malID int, episode string) []domain.SkipSegment {
if malID <= 0 || strings.TrimSpace(episode) == "" {
return []domain.SkipSegment{}
}
segments := []domain.SkipSegment{}
endpoint := fmt.Sprintf("https://api.aniskip.com/v1/skip-times/%s/%s?types=op&types=ed", url.PathEscape(strconv.Itoa(malID)), url.PathEscape(episode))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err == nil {
req.Header.Set("User-Agent", netutil.Generic)
if resp, err := s.httpClient.Do(req); err == nil {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode == http.StatusOK {
if body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.KiB512)); err == nil {
type resultItem struct {
SkipType string `json:"skip_type"`
Interval struct {
StartTime float64 `json:"start_time"`
EndTime float64 `json:"end_time"`
} `json:"interval"`
}
type apiResponse struct {
Found bool `json:"found"`
Result []resultItem `json:"results"`
}
var parsed apiResponse
if err := json.Unmarshal(body, &parsed); err == nil && parsed.Found && len(parsed.Result) > 0 {
segments = make([]domain.SkipSegment, 0, len(parsed.Result))
for _, r := range parsed.Result {
skipType := strings.ToLower(r.SkipType)
switch skipType {
case "op":
skipType = "opening"
case "ed":
skipType = "ending"
}
segments = append(segments, domain.SkipSegment{
Type: skipType,
Start: r.Interval.StartTime,
End: r.Interval.EndTime,
Source: "aniskip",
})
}
}
}
}
}
}
epNum, _ := strconv.ParseInt(strings.TrimSpace(episode), 10, 64)
if userID != "" && epNum > 0 {
if ok, err := s.repo.HasSkipSegmentOverrideTable(ctx); err == nil && ok {
if overrides, err := s.repo.ListSkipSegmentOverrides(ctx, userID, int64(malID), epNum); err == nil {
// Build map keyed by normalized type ("opening"/"ending")
overrideByType := make(map[string]domain.SkipSegment, len(overrides))
for _, o := range overrides {
t := strings.ToLower(strings.TrimSpace(o.SkipType))
switch t {
case "op", "opening", "intro":
t = "opening"
case "ed", "ending", "outro":
t = "ending"
default:
continue
}
overrideByType[t] = domain.SkipSegment{
Type: t,
Start: o.StartTime,
End: o.EndTime,
Source: "override",
}
}
if len(overrideByType) > 0 {
merged := make([]domain.SkipSegment, 0, len(segments)+len(overrideByType))
seen := map[string]bool{}
for _, seg := range segments {
if o, ok := overrideByType[seg.Type]; ok {
merged = append(merged, o)
seen[seg.Type] = true
} else {
merged = append(merged, seg)
seen[seg.Type] = true
}
}
for t, o := range overrideByType {
if !seen[t] {
merged = append(merged, o)
}
}
segments = merged
}
}
}
}
return segments
}
func (s *playbackService) warmStreamURL(targetURL, referer string) {
req, err := http.NewRequest(http.MethodGet, targetURL, nil)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, targetURL, nil)
if err != nil {
return
}
@@ -542,10 +71,6 @@ func (s *playbackService) warmStreamURL(targetURL, referer string) {
}
req.Header.Set("User-Agent", netutil.Firefox121)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req = req.WithContext(ctx)
resp, err := s.httpClient.Do(req)
if err != nil {
return

210
internal/playback/skip_segments.go Normal file
View File

@@ -0,0 +1,210 @@
package playback
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"github.com/google/uuid"
"mal/internal/db"
"mal/internal/domain"
netutil "mal/pkg/net"
)
func normalizeSkipType(skipType string) (string, error) {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op", "opening", "intro":
return "op", nil
case "ed", "ending", "outro":
return "ed", nil
default:
return "", fmt.Errorf("invalid skip_type")
}
}
func (s *playbackService) UpsertSkipSegmentOverride(ctx context.Context, userID string, animeID int64, episode int, skipType string, startTime, endTime float64) error {
if userID == "" {
return fmt.Errorf("not authenticated")
}
if animeID <= 0 || episode <= 0 {
return fmt.Errorf("invalid anime/episode")
}
t, err := normalizeSkipType(skipType)
if err != nil {
return err
}
if !(startTime >= 0) || !(endTime > startTime) {
return fmt.Errorf("invalid interval")
}
if endTime-startTime < 5 || endTime-startTime > 10*60 {
return fmt.Errorf("interval duration out of range")
}
return s.repo.UpsertSkipSegmentOverride(ctx, db.SkipSegmentOverrideRow{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Episode: int64(episode),
SkipType: t,
StartTime: startTime,
EndTime: endTime,
})
}
func (s *playbackService) fetchSkipSegments(ctx context.Context, userID string, malID int, episode string) []domain.SkipSegment {
if malID <= 0 || strings.TrimSpace(episode) == "" {
return []domain.SkipSegment{}
}
segments := s.fetchAniSkipSegments(ctx, malID, episode)
return s.applySkipSegmentOverrides(ctx, segments, userID, malID, episode)
}
func (s *playbackService) fetchAniSkipSegments(ctx context.Context, malID int, episode string) []domain.SkipSegment {
endpoint := fmt.Sprintf("https://api.aniskip.com/v1/skip-times/%s/%s?types=op&types=ed", url.PathEscape(strconv.Itoa(malID)), url.PathEscape(episode))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return nil
}
req.Header.Set("User-Agent", netutil.Generic)
resp, err := s.httpClient.Do(req)
if err != nil {
return nil
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return nil
}
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.KiB512))
if err != nil {
return nil
}
return parseAniSkipSegments(body)
}
func parseAniSkipSegments(body []byte) []domain.SkipSegment {
type resultItem struct {
SkipType string `json:"skip_type"`
Interval struct {
StartTime float64 `json:"start_time"`
EndTime float64 `json:"end_time"`
} `json:"interval"`
}
type apiResponse struct {
Found bool `json:"found"`
Result []resultItem `json:"results"`
}
var parsed apiResponse
if err := json.Unmarshal(body, &parsed); err != nil || !parsed.Found || len(parsed.Result) == 0 {
return nil
}
segments := make([]domain.SkipSegment, 0, len(parsed.Result))
for _, item := range parsed.Result {
segments = append(segments, domain.SkipSegment{
Type: normalizeSkipSegmentLabel(item.SkipType),
Start: item.Interval.StartTime,
End: item.Interval.EndTime,
Source: "aniskip",
})
}
return segments
}
func normalizeSkipSegmentLabel(skipType string) string {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op":
return "opening"
case "ed":
return "ending"
default:
return strings.ToLower(strings.TrimSpace(skipType))
}
}
func (s *playbackService) applySkipSegmentOverrides(ctx context.Context, segments []domain.SkipSegment, userID string, malID int, episode string) []domain.SkipSegment {
epNum, err := strconv.ParseInt(strings.TrimSpace(episode), 10, 64)
if userID == "" || err != nil || epNum <= 0 {
return segments
}
ok, err := s.repo.HasSkipSegmentOverrideTable(ctx)
if err != nil || !ok {
return segments
}
overrides, err := s.repo.ListSkipSegmentOverrides(ctx, userID, int64(malID), epNum)
if err != nil {
return segments
}
overrideByType := buildOverrideSegments(overrides)
if len(overrideByType) == 0 {
return segments
}
return mergeSkipSegments(segments, overrideByType)
}
func buildOverrideSegments(overrides []db.SkipSegmentOverrideRow) map[string]domain.SkipSegment {
byType := make(map[string]domain.SkipSegment, len(overrides))
for _, override := range overrides {
skipType, ok := normalizeOverrideSkipType(override.SkipType)
if !ok {
continue
}
byType[skipType] = domain.SkipSegment{
Type: skipType,
Start: override.StartTime,
End: override.EndTime,
Source: "override",
}
}
return byType
}
func normalizeOverrideSkipType(skipType string) (string, bool) {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op", "opening", "intro":
return "opening", true
case "ed", "ending", "outro":
return "ending", true
default:
return "", false
}
}
func mergeSkipSegments(segments []domain.SkipSegment, overrides map[string]domain.SkipSegment) []domain.SkipSegment {
merged := make([]domain.SkipSegment, 0, len(segments)+len(overrides))
seen := make(map[string]bool, len(segments))
for _, segment := range segments {
if override, ok := overrides[segment.Type]; ok {
merged = append(merged, override)
} else {
merged = append(merged, segment)
}
seen[segment.Type] = true
}
for skipType, override := range overrides {
if !seen[skipType] {
merged = append(merged, override)
}
}
return merged
}

228
internal/playback/watch_data.go Normal file
View File

@@ -0,0 +1,228 @@
package playback
import (
"context"
"fmt"
"sort"
"strconv"
"strings"
"mal/integrations/jikan"
"mal/internal/domain"
)
func (s *playbackService) BuildWatchData(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string, userID string) (domain.WatchPageData, error) {
anime, err := s.jikan.GetAnimeByID(ctx, animeID)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch anime: %w", err)
}
animeData := domain.Anime{Anime: anime}
s.ensureAnimeRow(ctx, animeData)
searchTitles := buildSearchTitles(animeData, titleCandidates)
canonicalEpisodes, err := s.episodes.GetCanonicalEpisodes(ctx, animeData, false)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch episodes: %w", err)
}
mode, modeSwitchedFrom := resolveMode(episode, mode, canonicalEpisodes.Episodes)
modeSources, result, resolvedMode, resolvedModeSwitchedFrom := s.resolveModeSources(ctx, animeID, searchTitles, episode, mode)
if resolvedMode != "" {
mode = resolvedMode
}
if resolvedModeSwitchedFrom != "" {
modeSwitchedFrom = resolvedModeSwitchedFrom
}
if len(modeSources) == 0 {
return domain.WatchPageData{}, fmt.Errorf("no streams found")
}
if result == nil {
return domain.WatchPageData{}, fmt.Errorf("no streams found for mode %s", mode)
}
startTime, watchlistStatus, watchlistIDs := s.loadWatchProgress(ctx, userID, animeID, anime.Episodes, episode)
go s.warmStreamURL(result.URL, result.Referer)
seasons := s.loadSeasons(ctx, animeID)
segments := s.fetchSkipSegments(ctx, userID, animeID, episode)
watchData := buildWatchDataPayload(animeData, animeID, episode, startTime, canonicalEpisodes.Episodes, modeSources, mode, modeSwitchedFrom, segments)
return buildWatchPageData(animeData, canonicalEpisodes.Episodes, episode, watchlistStatus, watchlistIDs, seasons, watchData), nil
}
func buildWatchDataPayload(anime domain.Anime, animeID int, episode string, startTime float64, episodes []domain.CanonicalEpisode, modeSources map[string]domain.ModeSource, mode string, modeSwitchedFrom string, segments []domain.SkipSegment) domain.WatchData {
return domain.WatchData{
MalID: animeID,
Title: anime.DisplayTitle(),
CurrentEpisode: episode,
StartTimeSeconds: startTime,
Episodes: episodes,
Providers: []domain.ProviderData{{Streams: []domain.ProviderStream{{
Name: "Primary",
Quality: "Auto",
MalID: animeID,
IsCurrent: true,
}}}},
ModeSources: modeSources,
InitialMode: mode,
ModeSwitchedFrom: modeSwitchedFrom,
AvailableModes: availableModes(modeSources),
Segments: segments,
Airing: anime.Airing,
}
}
func buildWatchPageData(anime domain.Anime, episodes []domain.CanonicalEpisode, episode string, watchlistStatus string, watchlistIDs []int64, seasons []domain.SeasonEntry, watchData domain.WatchData) domain.WatchPageData {
return domain.WatchPageData{
WatchData: watchData,
Anime: anime,
Episodes: episodes,
CurrentEpID: episode,
WatchlistStatus: watchlistStatus,
WatchlistIDs: watchlistIDs,
Seasons: seasons,
}
}
func buildSearchTitles(anime domain.Anime, titleCandidates []string) []string {
seen := map[string]struct{}{}
out := make([]string, 0, 3+len(anime.TitleSynonyms)+len(titleCandidates))
appendTitle := func(title string) {
title = strings.TrimSpace(title)
if title == "" {
return
}
if _, ok := seen[title]; ok {
return
}
seen[title] = struct{}{}
out = append(out, title)
}
appendTitle(anime.Title)
appendTitle(anime.TitleEnglish)
appendTitle(anime.TitleJapanese)
for _, syn := range anime.TitleSynonyms {
appendTitle(syn)
}
for _, candidate := range titleCandidates {
appendTitle(candidate)
}
return out
}
func resolveMode(episode string, requestedMode string, episodes []domain.CanonicalEpisode) (string, string) {
if requestedMode != "dub" {
return requestedMode, ""
}
epNum, err := strconv.Atoi(episode)
if err != nil {
return requestedMode, ""
}
for _, ep := range episodes {
if ep.Number == epNum && !ep.HasDub && ep.HasSub {
return "sub", requestedMode
}
}
return requestedMode, ""
}
func (s *playbackService) resolveModeSources(ctx context.Context, animeID int, searchTitles []string, episode string, requestedMode string) (map[string]domain.ModeSource, *domain.StreamResult, string, string) {
if res := s.resolveStreamResult(ctx, animeID, searchTitles, episode, requestedMode); res != nil {
return map[string]domain.ModeSource{
requestedMode: s.buildModeSource(res),
}, res, requestedMode, ""
}
for _, fallbackMode := range fallbackModes(requestedMode) {
res := s.resolveStreamResult(ctx, animeID, searchTitles, episode, fallbackMode)
if res == nil {
continue
}
return map[string]domain.ModeSource{
fallbackMode: s.buildModeSource(res),
}, res, fallbackMode, requestedMode
}
return map[string]domain.ModeSource{}, nil, requestedMode, ""
}
func (s *playbackService) resolveStreamResult(ctx context.Context, animeID int, searchTitles []string, episode string, mode string) *domain.StreamResult {
for _, p := range s.providers {
res, err := p.GetStreams(ctx, animeID, searchTitles, episode, mode)
if err == nil && res != nil {
return res
}
}
return nil
}
func (s *playbackService) buildModeSource(res *domain.StreamResult) domain.ModeSource {
subtitles := make([]domain.SubtitleItem, 0, len(res.Subtitles))
for _, sub := range res.Subtitles {
token, _ := s.SignProxyToken(sub.URL, res.Referer, "subtitle")
subtitles = append(subtitles, domain.SubtitleItem{
Lang: sub.Label,
Token: token,
})
}
streamToken, _ := s.SignProxyToken(res.URL, res.Referer, "stream")
return domain.ModeSource{
Token: streamToken,
Type: res.Type,
Subtitles: subtitles,
}
}
func (s *playbackService) loadSeasons(ctx context.Context, animeID int) []domain.SeasonEntry {
relations, _ := s.jikan.GetFullRelations(ctx, animeID, jikan.WatchOrderModeMain)
seasons := make([]domain.SeasonEntry, 0, len(relations))
tvCounter := 1
for _, rel := range relations {
animeType := strings.ToLower(rel.Anime.Type)
if animeType != "tv" && animeType != "movie" {
continue
}
season := domain.SeasonEntry{
MalID: rel.Anime.MalID,
Title: rel.Anime.DisplayTitle(),
Prefix: rel.Relation,
IsCurrent: rel.IsCurrent,
}
if rel.Relation == "TV" {
season.Prefix = fmt.Sprintf("S%d", tvCounter)
tvCounter++
}
seasons = append(seasons, season)
}
return seasons
}
func availableModes(modeSources map[string]domain.ModeSource) []string {
modes := make([]string, 0, len(modeSources))
for mode := range modeSources {
modes = append(modes, mode)
}
sort.Strings(modes)
return modes
}
func fallbackModes(requestedMode string) []string {
switch requestedMode {
case "sub":
return []string{"dub"}
case "dub":
return []string{"sub"}
default:
return []string{"sub", "dub"}
}
}

35
internal/playback/watch_data_test.go Normal file
View File

@@ -0,0 +1,35 @@
package playback
import (
"testing"
)
func TestFallbackModes(t *testing.T) {
t.Parallel()
tests := []struct {
name string
mode string
want []string
}{
{name: "sub falls back to dub", mode: "sub", want: []string{"dub"}},
{name: "dub falls back to sub", mode: "dub", want: []string{"sub"}},
{name: "unknown tries both canonical modes", mode: "raw", want: []string{"sub", "dub"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := fallbackModes(tt.mode)
if len(got) != len(tt.want) {
t.Fatalf("len(got) = %d, want %d", len(got), len(tt.want))
}
for i, want := range tt.want {
if got[i] != want {
t.Fatalf("got[%d] = %q, want %q", i, got[i], want)
}
}
})
}
}

4
internal/server/cors.go
View File

@@ -8,10 +8,6 @@ import (
"github.com/gin-gonic/gin"
)
func CORSMiddleware() gin.HandlerFunc {
return CORSMiddlewareWithConfig(config.Config{})
}
func CORSMiddlewareWithConfig(cfg config.Config) gin.HandlerFunc {
allowAll := cfg.CORSAllowAll
return func(c *gin.Context) {

45
internal/server/observability.go
View File

@@ -31,23 +31,40 @@ func RequestLogger(metrics *observability.Metrics) gin.HandlerFunc {
level = observability.LogLevelWarn
}
observability.LogJSON(
fields := map[string]any{
"client_ip": c.ClientIP(),
"duration_ms": float64(duration.Microseconds()) / 1000,
"method": c.Request.Method,
"path": path,
"request_id": c.Writer.Header().Get(requestIDHeader),
"status": status,
}
privateErrors := c.Errors.ByType(gin.ErrorTypePrivate)
var logErr error
if len(privateErrors) > 0 {
logErr = privateErrors.Last().Err
}
if route != path {
fields["route"] = route
}
if query != "" {
fields["query"] = query
}
if size := c.Writer.Size(); size >= 0 {
fields["bytes"] = size
}
if errors := privateErrors.String(); errors != "" {
fields["errors"] = errors
}
observability.LogContext(
c.Request.Context(),
level,
"http_request",
"http",
"",
map[string]any{
"method": c.Request.Method,
"route": route,
"path": path,
"query": query,
"status": status,
"duration_ms": float64(duration.Microseconds()) / 1000,
"bytes": c.Writer.Size(),
"client_ip": c.ClientIP(),
"errors": c.Errors.ByType(gin.ErrorTypePrivate).String(),
},
nil,
c.Request.Method+" "+path,
fields,
logErr,
)
}
}

30
internal/server/request_context.go Normal file
View File

@@ -0,0 +1,30 @@
package server
import (
"mal/internal/observability"
"strings"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
)
const requestIDHeader = "X-Request-ID"
func RequestContextMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
requestID := strings.TrimSpace(c.GetHeader(requestIDHeader))
if requestID == "" {
requestID = uuid.NewString()
}
path := c.Request.URL.Path
route := c.FullPath()
if route == "" {
route = path
}
c.Writer.Header().Set(requestIDHeader, requestID)
c.Request = c.Request.WithContext(observability.WithRequestContext(c.Request.Context(), requestID, path, route))
c.Next()
}
}

13
internal/server/respond.go
View File

@@ -27,7 +27,18 @@ func RespondError(c *gin.Context, status int, event string, component string, me
if status >= http.StatusInternalServerError {
level = observability.LogLevelError
}
observability.LogJSON(level, event, component, "", fields, err)
if fields == nil {
fields = make(map[string]any, 2)
}
if _, exists := fields["request_path"]; !exists {
fields["request_path"] = c.Request.URL.Path
}
if route := c.FullPath(); route != "" && route != c.Request.URL.Path {
if _, exists := fields["request_route"]; !exists {
fields["request_route"] = route
}
}
observability.LogContext(c.Request.Context(), level, event, component, "", fields, err)
RespondHTMLOrJSONError(c, status, message)
}

2
internal/server/server.go
View File

@@ -27,7 +27,7 @@ func ProvideRouter(cfg config.Config, htmlRender render.HTMLRender, metrics *obs
gin.SetMode(cfg.GinMode)
}
r := gin.New()
r.Use(CORSMiddlewareWithConfig(cfg), audit.ContextMiddleware(), RequestLogger(metrics), gin.Recovery())
r.Use(CORSMiddlewareWithConfig(cfg), RequestContextMiddleware(), audit.ContextMiddleware(), RequestLogger(metrics), gin.Recovery())
r.Static("/static", "./static")
r.Static("/dist", "./dist")
r.GET("/metrics", gin.WrapH(metrics.Handler()))

55
internal/server/server_test.go
View File

@@ -2,6 +2,7 @@ package server
import (
"bytes"
"context"
"io"
"log"
"mal/internal/observability"
@@ -43,12 +44,13 @@ func TestRequestLoggerUsesMatchedRoute(t *testing.T) {
defer log.SetOutput(previousOutput)
router := gin.New()
router.Use(RequestContextMiddleware())
router.Use(RequestLogger(observability.NewMetrics()))
router.GET("/anime/:id", func(c *gin.Context) {
c.String(http.StatusOK, "ok")
})
req := httptest.NewRequest(http.MethodGet, "/anime/1?section=characters", nil)
req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/anime/1?section=characters", nil)
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
@@ -58,13 +60,54 @@ func TestRequestLoggerUsesMatchedRoute(t *testing.T) {
}
logLine := string(output)
if !strings.Contains(logLine, `"event":"http_request"`) {
t.Fatalf("log line missing event: %s", logLine)
if !strings.Contains(logLine, " INFO http 200 GET /anime/1") {
t.Fatalf("log line missing compact http summary: %s", logLine)
}
if !strings.Contains(logLine, `"route":"/anime/:id"`) {
if !strings.Contains(logLine, " route=/anime/:id") {
t.Fatalf("log line missing route: %s", logLine)
}
if !strings.Contains(logLine, `"status":200`) {
t.Fatalf("log line missing status: %s", logLine)
if !strings.Contains(logLine, " request_id=") {
t.Fatalf("log line missing request id: %s", logLine)
}
if strings.Contains(logLine, `"GET /anime/1"`) {
t.Fatalf("log line should not duplicate request summary: %s", logLine)
}
if rec.Header().Get(requestIDHeader) == "" {
t.Fatalf("expected %s response header to be set", requestIDHeader)
}
}
func TestRespondErrorIncludesRequestContext(t *testing.T) {
gin.SetMode(gin.TestMode)
var logs bytes.Buffer
previousOutput := log.Writer()
log.SetOutput(&logs)
defer log.SetOutput(previousOutput)
router := gin.New()
router.Use(RequestContextMiddleware())
router.GET("/anime/:id", func(c *gin.Context) {
RespondError(c, http.StatusInternalServerError, "anime_lookup_failed", "anime", "failed", nil, context.DeadlineExceeded)
})
req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/anime/1", nil)
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
output, err := io.ReadAll(&logs)
if err != nil {
t.Fatalf("read logs: %v", err)
}
logLine := string(output)
if !strings.Contains(logLine, " request_id=") {
t.Fatalf("log line missing request id: %s", logLine)
}
if !strings.Contains(logLine, " request_path=/anime/1") {
t.Fatalf("log line missing request path: %s", logLine)
}
if !strings.Contains(logLine, " request_route=/anime/:id") {
t.Fatalf("log line missing request route: %s", logLine)
}
}

9
justfile
View File

@@ -17,14 +17,13 @@ test:
go test ./...
build-go:
go build -o server ./cmd/server
@go build -o server ./cmd/server
build-css:
bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css
@bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css
build-ts:
bun build ./static/player/main.ts --outdir ./dist/static/player --target browser --splitting
bun build ./static/*.ts --outdir ./dist/static --target browser --root ./static --entry-naming "[name].js"
@bun ./scripts/build-ts.ts
build: build-go build-css build-ts
@@ -37,7 +36,7 @@ install-hooks:
bunx lefthook install
dev: build
./server
@./server
clean:
rm -rf dist/*

41
lefthook.yml
View File

@@ -2,17 +2,42 @@
"$schema": "https://json.schemastore.org/lefthook.json",
"pre-commit":
{
"fail_on_changes": "always",
"fail_on_changes_diff": true,
"commands":
{
"format": { "run": "bunx oxfmt" },
"format":
{
"glob": "*.{ts,js,tsx,jsx,css,json,html}",
"run": "bunx oxfmt --check {staged_files}",
},
"lint:ts":
{ "run": "bunx oxlint --ignore-path .oxlintignore static --max-warnings 0 --fix" },
"go-fmt": { "run": "go fmt ./..." },
"go-lint": { "run": "bun run lint:go" },
"go-test": { "run": "go test ./..." },
"ts-typecheck": { "run": "bunx tsc -p tsconfig.json --noEmit" },
"build-assets": { "run": "bun run build:assets" },
"go-build": { "run": "go build -o server ./cmd/server" },
{
"glob": "*.{ts,js,tsx,jsx}",
"run": "bunx oxlint --ignore-path .oxlintignore {staged_files} --max-warnings 0 --tsconfig ./tsconfig.json --type-aware",
},
"go-fmt":
{
"glob": "*.go",
"run": 'files=$(gofmt -l {staged_files}); test -z "$files" || (printf "go files need formatting:\n%s\n" "$files"; exit 1)',
},
"go-lint":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} golangci-lint run --new-from-rev=HEAD ./{}',
},
"go-test":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} go test -count=1 ./{}',
},
"ts-typecheck": { "glob": "*.ts", "run": "bunx tsc -p tsconfig.json --noEmit" },
"build-assets": { "glob": "*.{ts,css}", "run": "bun run build:assets" },
"go-build":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} go build -o /dev/null ./{}',
},
},
},
}

7
package.json
View File

@@ -3,9 +3,9 @@
"private": true,
"type": "module",
"scripts": {
"build:css": "bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css",
"watch:css": "bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css --watch",
"build:ts": "bun build ./static/player/main.ts --outdir ./dist/static/player --target browser --splitting && bun build ./static/*.ts --outdir ./dist/static --target browser --root ./static --entry-naming \"[name].js\" && cp ./node_modules/htmx.org/dist/htmx.min.js ./dist/static/htmx-lib.js",
"build:css": "bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css",
"watch:css": "bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css --watch",
"build:ts": "bun ./scripts/build-ts.ts",
"typecheck": "bunx tsc -p tsconfig.json --noEmit",
"build:assets": "bun run build:css && bun run build:ts",
"format": "bunx oxfmt",
@@ -16,6 +16,7 @@
"lint:go": "golangci-lint run ./..."
},
"dependencies": {
"hls.js": "^1.6.16",
"htmx.org": "1.9.12"
},
"devDependencies": {

8
pkg/graphql.go
View File

@@ -53,12 +53,12 @@ func Post[T any](ctx context.Context, httpClient *http.Client, url string, query
}
defer func() { _ = resp.Body.Close() }()
max := opts.BodyMax
if max <= 0 {
max = 2 << 20
bodyMax := opts.BodyMax
if bodyMax <= 0 {
bodyMax = 2 << 20
}
respBody, err := io.ReadAll(io.LimitReader(resp.Body, max))
respBody, err := io.ReadAll(io.LimitReader(resp.Body, bodyMax))
if err != nil {
return zero, fmt.Errorf("graphql: read response: %w", err)
}

22
pkg/net/document.go
View File

@@ -9,13 +9,23 @@ import (
"github.com/PuerkitoBio/goquery"
)
func responseURL(response *http.Response, fallbackRequest *http.Request) string {
if response != nil && response.Request != nil && response.Request.URL != nil {
return response.Request.URL.String()
}
if fallbackRequest != nil && fallbackRequest.URL != nil {
return fallbackRequest.URL.String()
}
return ""
}
func FetchHTMLDocument(
ctx context.Context,
httpClient *http.Client,
url string,
prepareRequest func(*http.Request),
buildStatusError func(*http.Response, []byte) error,
) (*goquery.Document, *http.Response, error) {
) (*goquery.Document, string, error) {
client := httpClient
if client == nil {
client = http.DefaultClient
@@ -23,7 +33,7 @@ func FetchHTMLDocument(
request, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, nil, fmt.Errorf("failed to create request: %w", err)
return nil, "", fmt.Errorf("failed to create request: %w", err)
}
if prepareRequest != nil {
prepareRequest(request)
@@ -31,19 +41,19 @@ func FetchHTMLDocument(
response, err := client.Do(request)
if err != nil {
return nil, nil, fmt.Errorf("request failed: %w", err)
return nil, "", fmt.Errorf("request failed: %w", err)
}
defer func() { _ = response.Body.Close() }()
if response.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(response.Body, Bytes512))
return nil, response, buildStatusError(response, body)
return nil, responseURL(response, request), buildStatusError(response, body)
}
document, err := goquery.NewDocumentFromReader(response.Body)
if err != nil {
return nil, response, fmt.Errorf("failed to parse html: %w", err)
return nil, responseURL(response, request), fmt.Errorf("failed to parse html: %w", err)
}
return document, response, nil
return document, responseURL(response, request), nil
}

41
pkg/net/document_test.go Normal file
View File

@@ -0,0 +1,41 @@
package netutil
import (
"context"
"io"
"net/http"
"strings"
"testing"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) {
return f(request)
}
func TestFetchHTMLDocumentFallsBackToOriginalURLWhenResponseRequestMissing(t *testing.T) {
client := &http.Client{
Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader("<!doctype html><html><body><main>ok</main></body></html>")),
}, nil
}),
}
url := "https://example.test/watch-order"
document, finalURL, err := FetchHTMLDocument(context.Background(), client, url, nil, nil)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if finalURL != url {
t.Fatalf("expected final url %q, got %q", url, finalURL)
}
if got := strings.TrimSpace(document.Find("main").Text()); got != "ok" {
t.Fatalf("expected document text ok, got %q", got)
}
}

91
scripts/build-ts.ts Normal file
View File

@@ -0,0 +1,91 @@
import { copyFile } from "node:fs/promises";
import { readdirSync } from "node:fs";
import { spawnSync } from "node:child_process";
type BuildStep = {
name: string;
command: string[];
};
const steps: BuildStep[] = [
{
name: "player",
command: [
"bun",
"build",
"./static/player/main.ts",
"--outdir",
"./dist/static/player",
"--target",
"browser",
"--splitting",
],
},
];
const startedAt = performance.now();
main().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
console.error(`ts build failed: ${message}`);
process.exit(1);
});
async function main(): Promise<void> {
const appEntries = readdirSync("./static", { withFileTypes: true })
.filter((entry) => entry.isFile() && entry.name.endsWith(".ts"))
.map((entry) => `./static/${entry.name}`)
.sort();
steps.push({
name: "app",
command: [
"bun",
"build",
...appEntries,
"--outdir",
"./dist/static",
"--target",
"browser",
"--root",
"./static",
"--entry-naming",
"[name].js",
],
});
for (const step of steps) {
const result = spawnSync(step.command[0], step.command.slice(1), {
stdio: "pipe",
});
if (result.status !== 0) {
const detail = summarizeFailure(result.stderr, result.stdout);
console.error(`ts build failed at ${step.name}${detail === "" ? "" : `: ${detail}`}`);
process.exit(result.status ?? 1);
}
}
await copyFile("./node_modules/htmx.org/dist/htmx.min.js", "./dist/static/htmx-lib.js");
const playerEntries = 1;
const totalEntries = playerEntries + appEntries.length;
const elapsedMs = Math.round(performance.now() - startedAt);
console.log(`ts build ok (${totalEntries} entries, ${elapsedMs}ms)`);
}
function summarizeFailure(stderr: Uint8Array, stdout: Uint8Array): string {
const combined =
`${Buffer.from(stderr).toString("utf8")}${Buffer.from(stdout).toString("utf8")}`.trim();
if (combined === "") {
return "";
}
const lines = combined
.split(/\r?\n/)
.map((line) => line.trim())
.filter((line) => line !== "");
return lines[lines.length - 1] ?? "";
}

3
static/app.ts
View File

@@ -2,15 +2,12 @@ import "./theme";
import "./toast";
import "./htmx";
import "./dropdown";
import "./discover";
import "./anime";
import "./timezone";
import "./search";
import "./sort_filter";
import "./dedupe";
import "./shell";
import "./watchlist";
import "./top_pick_carousel";
import "./continue_watching_carousel";
import "./login";
import "./schedule";

BIN
static/assets/app-icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 80 KiB

18
static/assets/app-icon.svg
View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 608 B

18
static/assets/apple-touch-icon-120x120-precomposed.svg
View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 608 B

BIN
static/assets/apple-touch-icon-120x120.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.3 KiB

18
static/assets/apple-touch-icon-120x120.svg
View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 608 B

18
static/assets/apple-touch-icon-precomposed.svg
View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 608 B

BIN
static/assets/apple-touch-icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

18
static/assets/apple-touch-icon.svg
View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 608 B

BIN
static/assets/favicon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 113 KiB

23
static/assets/favicon.svg
View File

@@ -1,23 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
<clipPath id="clip">
<circle cx="50" cy="50" r="45" />
</clipPath>
</defs>
<!-- Base -->
<circle cx="50" cy="50" r="45" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<g clip-path="url(#clip)">
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</g>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 685 B

BIN
static/assets/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 392 KiB

Some files were not shown because too many files have changed in this diff Show More