Compare commits

...

382 Commits

Author SHA1 Message Date
8fd7c1104c Merge branch 'upstream/main' into main
All checks were successful
Build and Push Container Image / build-and-push (push) Successful in 9m21s
2026-06-15 21:37:41 +02:00
6841f5c55a ci: drop sqlc from Docker build
sqlc-generated code is committed, no need to install and run it
during docker image builds
2026-06-14 21:58:04 +02:00
3e100c1a97 feat: ensure anime row exists before saving progress 2026-06-14 21:53:41 +02:00
4a74fdcf31 feat: add cache busting and hls query param 2026-06-14 21:51:02 +02:00
f9f3322797 feat: add hls.js for m3u8 stream playback 2026-06-14 21:37:55 +02:00
c891382efb feat: add type field to ModeSource and pass to loadVideoSource 2026-06-14 21:37:49 +02:00
ef36578c4b feat: propagate stream source type from provider to client 2026-06-14 21:37:38 +02:00
20aadd36f8 feat: preload alternate mode source on episode load 2026-06-14 21:19:59 +02:00
5dcf39c401 test: add fallbackModes unit tests 2026-06-14 21:17:33 +02:00
7b56f587e5 test: add parseOKRUSources unit test 2026-06-14 21:17:30 +02:00
43d31865ed test: add test for embed source skipping in resolveDirectSource 2026-06-14 21:17:27 +02:00
3668ccb541 refactor: wire mode fallback into resolveModeSources 2026-06-14 21:17:11 +02:00
7bf0ffbd06 feat: add fallbackModes helper 2026-06-14 21:17:09 +02:00
08a16f3302 feat: detect embeds in source references and route to extraction 2026-06-14 21:16:38 +02:00
dcebe90620 feat: add embed video parsing helpers for allanime 2026-06-14 21:16:28 +02:00
d28b187ac0 chore: update allanime site url and referer constants 2026-06-14 21:16:10 +02:00
c57ecf3d4b fix: skip error log on client disconnect in proxy handlers 2026-06-13 22:38:51 +02:00
d2528ba4f1 refactor: reduce search.ts to entry point 2026-06-13 22:29:34 +02:00
c8112e5062 feat: add search/overlay.ts 2026-06-13 22:29:30 +02:00
0d7c572f2c feat: add search/actions.ts 2026-06-13 22:29:26 +02:00
5dbb04dbdd feat: add search/fetch.ts 2026-06-13 22:29:23 +02:00
ff1cd7ce4a feat: add search/render.ts 2026-06-13 22:29:20 +02:00
4ac155c8cc feat: add search/state.ts 2026-06-13 22:29:16 +02:00
e3d82389e4 trim: keep only entrypoint in client.go 2026-06-13 22:24:10 +02:00
f99b30bf43 extract: add stream source resolution 2026-06-13 22:24:06 +02:00
21a1965fdd extract: add availability parsing 2026-06-13 22:24:02 +02:00
fdb79633df extract: add search and provider-id resolution 2026-06-13 22:23:58 +02:00
4876995652 extract: add decrypt and deobfuscation helpers 2026-06-13 22:23:53 +02:00
40be6d3132 refactor: add moved recommendation types to recommendations.go 2026-06-13 22:14:57 +02:00
6a256a20c5 refactor: strip recommendation code from service.go 2026-06-13 22:14:52 +02:00
9e8fb5c033 extract: add cache store, trim to orchestration 2026-06-13 22:12:08 +02:00
84a967856b extract: add provider mapping cache 2026-06-13 22:12:04 +02:00
639f8f424f extract: add refresh and broadcast policy 2026-06-13 22:12:01 +02:00
9fcdd36c5e extract: add merge/validation functions 2026-06-13 22:11:57 +02:00
04c0b8d601 refactor: extract progress and completion 2026-06-13 22:07:52 +02:00
b578bd661e refactor: extract skip segments handling 2026-06-13 22:07:21 +02:00
e2d9ecfb03 refactor: extract watch data building 2026-06-13 22:06:38 +02:00
d6f1c37ac3 refactor: extract proxy token store 2026-06-13 22:06:07 +02:00
837b99bc58 refactor: extract anime reviews handler 2026-06-13 21:54:05 +02:00
e1ddd59417 refactor: extract anime details handlers 2026-06-13 21:54:01 +02:00
ec5a17c392 refactor: extract browse and search handlers 2026-06-13 21:53:57 +02:00
19c5f7ef1f refactor: extract catalog and search handlers 2026-06-13 21:53:52 +02:00
5a703bc323 style: clean up top picks header and page 2026-06-13 21:39:19 +02:00
1a65ef2a9c style: remove extra newline 2026-06-13 21:32:51 +02:00
263bfafd04 style: use http status constant 2026-06-13 21:32:23 +02:00
7523215a71 style: fix linter nits 2026-06-13 21:31:42 +02:00
ea411e5feb perf: preallocate fetchedAnimes in fetchBaselineAnime 2026-06-13 21:30:49 +02:00
aced7bb5d9 refactor: replace wrapper lambda with direct function reference 2026-06-13 21:29:21 +02:00
195d8c0e60 refactor: replace inline lambda with NewPlaybackService 2026-06-13 21:28:42 +02:00
bcc75467f0 style: replace len(status) == 0 with status == '' 2026-06-13 21:27:51 +02:00
a922953776 refactor: replace wrapper lambda with direct function reference 2026-06-13 21:27:20 +02:00
bcd4106dce refactor: replace wrapper lambda with direct function reference 2026-06-13 21:26:32 +02:00
b519706429 refactor: simplify fx.WithLogger call 2026-06-13 21:25:38 +02:00
6ac9e38423 refactor: update top picks link from /discover/top-picks to /top-picks 2026-06-13 21:23:23 +02:00
e13022c7d4 refactor: add top picks link to header, remove discover and schedule nav 2026-06-13 21:23:20 +02:00
c82d25d9c8 feat: add standalone top picks template 2026-06-13 21:23:17 +02:00
55bd2a2582 refactor: remove schedule page template 2026-06-13 21:23:15 +02:00
443dbeda77 refactor: remove discover page template 2026-06-13 21:23:12 +02:00
b53a58905d refactor: remove discover and schedule script imports 2026-06-13 21:23:09 +02:00
e393d2759b refactor: remove schedule board client code 2026-06-13 21:23:05 +02:00
df4867c60d refactor: remove discover tab and surprise me client code 2026-06-13 21:23:01 +02:00
b281acdf88 refactor: remove schedule caching and ISO week helpers 2026-06-13 21:22:59 +02:00
ef1cd20f0b refactor: remove animeschedule.net integration 2026-06-13 21:22:56 +02:00
66faa1a13f refactor: replace schedule nav item with top picks 2026-06-13 21:22:52 +02:00
a0bfe9f889 refactor: remove AnimeDiscoverService fx registration 2026-06-13 21:22:47 +02:00
e44d64a651 refactor: remove AnimeDiscoverService and DiscoverSectionData 2026-06-13 21:22:44 +02:00
4256480e0c refactor: remove discover and schedule service methods 2026-06-13 21:22:40 +02:00
a976769cdd refactor: remove discover and schedule routes and handlers 2026-06-13 21:22:36 +02:00
8304c0a338 chore: enable additional golangci-lint linters 2026-06-13 21:22:29 +02:00
6918f0bf48 fix: prevent pre-commit hook leaks 2026-06-13 20:58:01 +02:00
997957a232 style: reformat isClosableDropdown guard 2026-06-13 20:52:12 +02:00
bd268ead10 fix: close more dropdown before opening segment modal 2026-06-13 20:51:52 +02:00
18f9ec2a95 refactor: switch watch layout to CSS grid 2026-06-13 20:48:26 +02:00
168fea8ab5 feat: add fullscreen overrides to video player 2026-06-13 20:48:23 +02:00
b4e2930112 docs: rewrite readme as minimal prose 2026-06-13 20:14:17 +02:00
370dec5f3b docs: update readme logo markup 2026-06-13 19:22:48 +02:00
b11af766f2 chore: update manifest for png icons and theme colors 2026-06-13 19:22:42 +02:00
7d55aa7837 chore: remove svg icon assets 2026-06-13 19:22:36 +02:00
fb03d73e66 chore: remove mobile menu shell module 2026-06-13 19:22:29 +02:00
012bfee03d refactor: replace sidebar with top header nav in base layout 2026-06-13 19:22:23 +02:00
c47ffcb5be feat: add png icon assets 2026-06-13 19:22:15 +02:00
b1afd2ef82 feat: add header navigation bar templates 2026-06-13 19:22:08 +02:00
f2213bd4aa feat: revamp schedule and home pages 2026-06-13 17:04:09 +02:00
70a6e9a6b5 refactor: remove discover page 2026-06-13 17:04:01 +02:00
9b7a2cac8f feat: add standalone search page 2026-06-13 16:27:14 +02:00
bf85c3b018 feat: add poster retry and dedupe to search 2026-06-13 16:26:44 +02:00
26a8878fc2 refactor: extract dedupeByID utility 2026-06-13 16:25:21 +02:00
a09ff85ff8 style: update player segment color to amber 2026-06-13 12:28:35 +02:00
c4a7151d99 chore: fix node_modules path exclusion in golangci config 2026-06-12 13:46:02 +02:00
4ba1944f70 style: apply formatter to search 2026-06-12 13:40:23 +02:00
45b4a01801 fix: handle nil response request in fetch document 2026-06-12 13:40:19 +02:00
ffe42a352b refactor: pass watch order mode in playback service 2026-06-12 13:40:06 +02:00
1f2fd4f53d refactor: add mode param to GetRelations interface 2026-06-12 13:40:03 +02:00
35a367d569 refactor: pass watch order mode and paginate command palette 2026-06-12 13:39:58 +02:00
36c0e87ae8 feat: add watch order mode toggle 2026-06-12 13:39:50 +02:00
18ed806fc0 fix: prefer original over japanese in db displaytitle 2026-06-12 13:17:55 +02:00
164232cf0d fix: prefer original over japanese in jikan displaytitle 2026-06-12 13:17:51 +02:00
ea587665f2 feat: colorize http status logs 2026-06-12 13:09:49 +02:00
fa88badc69 style: apply formatter changes 2026-06-12 11:39:00 +02:00
4c4c10b154 feat: redesign search overlay 2026-06-12 11:38:28 +02:00
97814b7223 refactor: streamline command palette results 2026-06-12 11:38:15 +02:00
c509144b30 refactor: use css variable for player segment color 2026-06-12 11:38:03 +02:00
ab9d585a1f fix: make build script typecheck 2026-06-12 11:37:54 +02:00
de9bcb5e40 refactor: make schedule board responsive with stacked layout 2026-06-12 11:01:55 +02:00
b607b091d5 feat: add section_action component 2026-06-12 11:01:53 +02:00
15ad54a847 refactor: polish dark theme colors and add border accents 2026-06-12 10:48:55 +02:00
3ae09d4014 refactor: remove browse link from navigation 2026-06-12 10:37:54 +02:00
90ae58b99e fix: change browse sort from desc to asc 2026-06-12 10:37:50 +02:00
c252739610 refactor: split LogEvent into smaller functions 2026-06-11 17:12:22 +02:00
3c2e6a6984 refactor: extract helpers to reduce formatHTTPRequestLog complexity 2026-06-11 17:12:19 +02:00
25471e0bd5 fix: replace nil context with context.TODO 2026-06-11 17:11:47 +02:00
ed90b5c7aa fix: remove nil error return 2026-06-11 14:51:57 +02:00
1485800c32 fix: use request context in server 2026-06-11 14:51:56 +02:00
faae7bc719 fix: use request context in metrics 2026-06-11 14:51:55 +02:00
acabd50970 fix: use execcontext in db 2026-06-11 14:49:57 +02:00
6ba387bb6a fix: use QueryRowContext in test to fix noctx lint 2026-06-11 14:48:51 +02:00
3d13cf9be8 fix: use context-aware db calls in cmd/user 2026-06-11 14:48:05 +02:00
7f05f026e9 refactor: split fetchSkipSegments and fix warmStreamURL noctx 2026-06-11 14:46:22 +02:00
2ed03a667b refactor: split BuildWatchData into focused helpers 2026-06-11 14:45:15 +02:00
2e79c32afe refactor: split getTopPicksForYou into focused helpers 2026-06-11 14:38:40 +02:00
7968fb57f6 refactor: split parseProviderResponse into smaller helpers 2026-06-11 14:35:31 +02:00
ba578d969a refactor: split seedRandomPool to reduce gocognit 2026-06-11 14:33:29 +02:00
5998b59e81 refactor: extract helpers from fetchWithRetry to reduce gocognit 2026-06-11 14:32:02 +02:00
c5acc63370 refactor: extract helpers from FetchWeek to reduce gocognit 2026-06-11 14:29:59 +02:00
b0769ddce7 refactor: shorten ProvideRenderer to satisfy funlen 2026-06-11 14:26:35 +02:00
1c86f802b4 chore: remove stale comment about sqlc name conflict in skip segment overrides 2026-06-11 14:24:18 +02:00
97dcb19b7d refactor: split long functions in episode service to fix funlen linter 2026-06-11 14:23:18 +02:00
8e4ce81232 refactor: extract helpers to reduce funlen in command_palette 2026-06-11 13:08:44 +02:00
f360e22beb refactor: extract scanContinueWatchingEntry helper 2026-06-11 13:06:35 +02:00
27c84a9603 style: gofmt alignment in init 2026-06-11 13:05:25 +02:00
a925cc069e refactor: shorten init below funlen threshold 2026-06-11 13:04:51 +02:00
76af597f4d refactor: shorten TestBuildSourceReferences below funlen threshold 2026-06-11 13:03:19 +02:00
0227c8688b refactor: extract duplicate table-test loop into helper 2026-06-11 13:02:06 +02:00
188eec58a2 refactor: reduce cyclomatic complexity of UpsertSkipSegmentOverride 2026-06-11 12:59:58 +02:00
233472b14d refactor: reduce cyclomatic complexity of mergeEpisodes 2026-06-11 12:58:47 +02:00
7265dec446 refactor: reduce cyclomatic complexity of AuthMiddleware 2026-06-11 12:57:26 +02:00
1ad3be5160 refactor: extract helpers to reduce cyclomatic complexity in audit test 2026-06-11 12:56:19 +02:00
01ee9b1022 refactor: reduce cyclomatic complexity of GetAiringSchedule 2026-06-11 12:53:59 +02:00
e77debb085 refactor: extract candidate score adjustments into helpers 2026-06-11 12:48:58 +02:00
c575bfae47 refactor: extract section path in anime details handler 2026-06-11 12:47:31 +02:00
0262f22876 refactor: reduce cyclomatic complexity of HandleBrowse 2026-06-11 12:45:54 +02:00
e04b11f97f refactor: reduce cyclomatic complexity of HandleProducers 2026-06-11 12:42:35 +02:00
55095791c7 refactor: reduce cyclomatic complexity of parseM3U8 2026-06-11 12:40:41 +02:00
983d805240 refactor: extract string slice helper in allanime client 2026-06-11 12:38:38 +02:00
0beec5fd56 refactor: reduce cyclomatic complexity of resolveSourceReferences 2026-06-11 12:36:35 +02:00
650fac1c90 refactor: reduce graphqlRequestWithHash complexity 2026-06-11 12:35:11 +02:00
870f8086e2 refactor: extract show resolution helpers from GetStreams 2026-06-11 12:33:12 +02:00
7af597d8fc refactor: reduce DurationSeconds complexity with token parsing 2026-06-11 12:31:33 +02:00
b72bace16a refactor: use url.Values in proxy token url 2026-06-11 12:29:29 +02:00
de939cc5f3 refactor: use url.Values in avatar url 2026-06-11 12:29:28 +02:00
4d6736a439 refactor: use url.Values in command palette search url 2026-06-11 12:29:27 +02:00
02bbc6c4d4 refactor: use url.Values in allanime graphql request 2026-06-11 12:29:23 +02:00
5ada1f72e4 feat: add shared query param helpers for jikan 2026-06-11 12:27:56 +02:00
4b95f85d4d chore: remove stray blank line in test 2026-06-11 12:19:59 +02:00
c36f02862d refactor: split getFullRelations into smaller helpers 2026-06-11 12:19:43 +02:00
704058a512 refactor: extract helpers from TestGetWithCacheReturnsStaleAndRefreshesAsync
Split setup (newTestCacheDB, insertCachedResponse) and async
polling (waitForFreshCache) out of the test to reduce its
cyclomatic complexity below 10. Switch DB calls to ExecContext
/ QueryRowContext to fix noctx lint.
2026-06-11 12:17:40 +02:00
9b19661fa3 refactor: extract skip/level helpers from logJikanCache 2026-06-11 12:15:00 +02:00
ca957b5cdc refactor: reduce cyclomatic complexity in fetchWeekAPI 2026-06-11 12:13:22 +02:00
03ccd54c85 refactor: extract parseAirType from parseMeta 2026-06-11 12:09:57 +02:00
c70adbd0ec fix: only report new lint issues with --new-from-rev 2026-06-11 12:06:04 +02:00
5f346d8dec fix: run linter at package level, not file level 2026-06-11 12:05:38 +02:00
3ade952653 fix: scope pre-commit hooks to staged files 2026-06-11 12:05:18 +02:00
37d7e0f6f0 chore: scope pre-commit hooks to staged files 2026-06-11 11:28:26 +02:00
f32bcf1288 fix: close response body in FetchHTMLDocument 2026-06-11 11:25:51 +02:00
7f98fbfa7a chore: remove unused CORSMiddleware wrapper 2026-06-11 11:18:05 +02:00
827b77cb20 fix: remove leading space in class attributes 2026-06-09 19:11:48 +02:00
b67727c21c test: add template function and renderer tests 2026-06-09 19:10:25 +02:00
470039d9e9 refactor: use posterURL in templates 2026-06-09 19:10:16 +02:00
ea518a7d0a refactor: simplify browseURL, add posterURL helper 2026-06-09 19:10:10 +02:00
bd89715ea0 chore: remove unused template files
- delete dropdown.gohtml (unused — codebase uses <ui-dropdown> directly)
- delete footer.gohtml (never referenced)
- update components/README.md to reflect actual files
2026-06-09 19:09:10 +02:00
49512a6708 refactor: replace scrollbar hacks with scrollbar-hidden class 2026-06-09 18:21:27 +02:00
070375eaa5 refactor: reorder head and use non-blocking font loading 2026-06-09 18:21:16 +02:00
1d4364d63e refactor: deduplicate sidebar navigation into data-driven loop 2026-06-09 18:21:08 +02:00
15876a4f86 refactor: consolidate css token system and add base utilities 2026-06-09 18:21:02 +02:00
1a35bd81bd fix: preserve schedule source items 2026-06-09 12:36:26 +02:00
21fd1110d4 feat: populate duration_seconds on anime upsert and add backfill fix 2026-06-08 02:26:56 +02:00
f8cf4579af test: add HLS playlist rewrite and detection tests 2026-06-08 02:13:41 +02:00
1a1189d035 feat: add HLS playlist rewriting to proxy stream 2026-06-08 02:13:32 +02:00
db4dc20603 refactor: replace HMAC proxy tokens with in-memory store 2026-06-08 02:13:21 +02:00
a4fa0beff5 refactor: update playback domain types and interfaces 2026-06-08 02:13:12 +02:00
39df0ff99a style: migrate watch page to v4 syntax 2026-06-07 17:45:56 +02:00
80a3481ebe style: migrate schedule page to v4 syntax 2026-06-07 17:45:48 +02:00
6efea21632 style: migrate index page to v4 syntax 2026-06-07 17:45:40 +02:00
4c90f759c9 style: migrate filter_bar and video_player to v4 syntax 2026-06-07 17:45:32 +02:00
470f9e3532 style: migrate anime page, watchlist_actions, and watchlist to v4 syntax 2026-06-07 17:45:24 +02:00
e355933ba8 style: migrate z-index/scrollbar in continue_watching, size shorthands in login 2026-06-07 17:45:08 +02:00
102317c9b0 style: migrate shadow variable syntax in dropdown component 2026-06-07 17:44:59 +02:00
cd7fab7fbd style: migrate z-index syntax in toast 2026-06-07 17:44:49 +02:00
7f6d2c82cb style: migrate important modifier syntax in browse and discover 2026-06-07 17:44:25 +02:00
bc9820c536 chore(deps): bump tailwindcss from 4.2.4 to 4.3.0 2026-06-07 17:43:51 +02:00
f90ff2e4c7 fix: update anime page layout 2026-06-06 17:54:56 +02:00
79be865989 fix: handle edge cases in continue watching carousel 2026-06-06 17:26:41 +02:00
18a335fd74 feat: add continue watching carousel 2026-06-06 17:26:22 +02:00
082219d2d4 test: add tests for mergeEpisodes capping and cache validation 2026-06-06 17:22:14 +02:00
b661b577dd feat: cap episode numbers to expected count and validate cached payload 2026-06-06 17:22:06 +02:00
fb6e48cf92 feat: add visual filler/recap indicator in episode list 2026-06-06 17:21:56 +02:00
a6cb71c65b refactor: move video source construction from inline script to initPlayer 2026-06-06 16:54:35 +02:00
e70574ac08 refactor: update anime page scripts 2026-06-06 16:54:27 +02:00
f9064b3b6c refactor: simplify dedupe module 2026-06-06 16:54:19 +02:00
4b1b4266d9 refactor: streamline mobile menu with event delegation 2026-06-06 16:54:11 +02:00
f7e7dfd161 feat: improve command palette focus management and aria 2026-06-06 16:54:03 +02:00
651db05cd0 feat: add htmx error toast on error class swap 2026-06-06 16:53:56 +02:00
470e5b092b refactor: read watchlist IDs from JSON script tag instead of global var 2026-06-06 16:53:48 +02:00
e06a20b5d0 refactor: switch watchlist IDs from global to JSON script tag 2026-06-06 16:53:40 +02:00
fe46dd9c48 refactor: replace inline theme dialog script with data attributes 2026-06-06 16:53:32 +02:00
cb8ef29cde refactor: replace inline scripts with module scripts block 2026-06-06 16:53:24 +02:00
03e741c561 refactor: use browseURL helper and simplify filter bar templates 2026-06-06 16:53:16 +02:00
9cb3e8fe27 test: add tests for browseURL helper 2026-06-06 16:53:08 +02:00
b9ca82dbd9 refactor: add browseURL template helper for filter URLs 2026-06-06 16:53:00 +02:00
5441b14737 feat: improve dropdown accessibility with aria and focus management 2026-06-06 16:52:52 +02:00
5cc03579b2 refactor: consolidate scripts into single app.js entry point 2026-06-06 16:52:22 +02:00
b5fc2dfe4e feat: add app entry point, password toggle, and schedule modules 2026-06-06 16:52:16 +02:00
78b36452ae refactor: migrate from htmx:afterSwap to onHtmxLoad 2026-06-06 16:51:12 +02:00
392bc10b99 refactor: replace DOMContentLoaded with onReady utility 2026-06-06 16:51:07 +02:00
5019e9fcb7 feat: add onHtmxLoad and closestFocusable utilities 2026-06-06 16:50:03 +02:00
4bcfc8fdb7 refactor: remove docs folder 2026-06-06 15:54:10 +02:00
b85b29aa13 feat: add top picks for you page 2026-06-06 13:34:18 +02:00
ede17ce8aa test: verify diversity reranker spreads repeated genres 2026-06-05 16:38:27 +02:00
9d964824dc feat: add multi-feature diversity reranker for recommendations 2026-06-05 16:38:19 +02:00
620434f61b feat: dedupe after htmx swap on swap target 2026-06-05 16:32:31 +02:00
6aeb887830 refactor: scope dedupe to parent container 2026-06-05 16:32:21 +02:00
24bc63e8e2 refactor: remove theme toggle from navigation 2026-06-05 16:24:17 +02:00
4791eebf48 refactor: remove theme toggle from footer 2026-06-05 16:24:12 +02:00
6b43fa7ce5 feat: add inline theme script to prevent FOUC 2026-06-05 16:24:07 +02:00
60ba1a4fb5 refactor: follow system color scheme via matchMedia listener 2026-06-05 16:23:59 +02:00
3ea5ea68ff refactor: remove unused htmx global type declaration 2026-06-05 16:23:53 +02:00
97623aad4d style: add color-scheme for light and dark themes 2026-06-05 16:23:50 +02:00
9587dd5a71 feat: add top pick for you section to homepage 2026-06-05 16:15:13 +02:00
8b26e5f036 test: add weighted taste profile and search query tests 2026-06-05 16:15:00 +02:00
b4061bc9b1 feat: integrate profile search into top pick service 2026-06-05 16:14:38 +02:00
e326f89d62 feat: add profile search query builders and weighted scoring 2026-06-05 16:14:28 +02:00
55ee13d4eb feat: timezone-aware schedule with browser tz and JST client conversion 2026-06-05 15:42:23 +02:00
356ac99c64 feat: show audio availability on anime detail page 2026-06-05 13:20:21 +02:00
9d58adea9c refactor: try sub and dub modes in allanime resolution, drop fallback 2026-06-05 13:20:12 +02:00
a8a53d2677 fix: polish watch page layout and button consistency 2026-06-04 16:37:06 +02:00
51ee38bb57 refactor: use recommendation engine in discover for-you 2026-06-04 16:10:15 +02:00
8ae79c301a feat: add recommendation scoring and reranking engine 2026-06-04 16:10:08 +02:00
c725d96035 docs: add recommendation architecture document 2026-06-04 16:09:53 +02:00
ede479c3e1 feat: add loading fragment templates and optimize section triggers 2026-06-04 11:28:34 +02:00
390f6386af feat: wire background warming for detail sections 2026-06-04 11:28:27 +02:00
3fe1135203 feat: warm anime recommendations in background 2026-06-04 11:28:20 +02:00
342bd096da feat: stale-while-revalidate cache for watch order 2026-06-04 11:28:13 +02:00
404fa3c406 feat: add htmx type declarations and process on ready 2026-06-04 11:28:06 +02:00
8b3bd30b6c feat: bundle htmx.org locally instead of loading from unpkg 2026-06-04 11:28:01 +02:00
0c4b35cc4b refactor: replace discover for-you swap with targeted htmx fragment 2026-06-04 11:00:40 +02:00
b639e933ff redesign: schedule page layout with scrollable calendar grid 2026-06-04 10:36:29 +02:00
59d903d400 refactor: consolidate skeleton styles into global css 2026-06-04 10:06:22 +02:00
4316ce3f1d test: add skip segment overrides table check 2026-06-03 09:10:28 +02:00
5604432187 refactor: share jst helpers 2026-06-01 22:32:12 +02:00
0483bc5cc1 refactor: dedupe scrub seek 2026-06-01 22:29:42 +02:00
983981a186 refactor: dedupe next nav 2026-06-01 22:28:49 +02:00
55bf11d8be refactor: share stream url 2026-06-01 22:26:57 +02:00
455490f07d refactor: share dom ready 2026-06-01 22:25:47 +02:00
36435b6eb5 refactor: dedupe html fetch 2026-06-01 22:24:27 +02:00
340daeadc6 refactor: dedupe html headers 2026-06-01 22:23:10 +02:00
625c3bbe25 refactor: dedupe repo tx 2026-06-01 22:22:14 +02:00
d5406a6857 refactor: dedupe jikan refresh 2026-06-01 22:21:19 +02:00
9f754012eb test: dedupe jikan bool cases 2026-06-01 22:20:12 +02:00
58036bea5a refactor: dedupe season fetch 2026-06-01 22:19:10 +02:00
25a8167461 refactor: dedupe anime warnings 2026-06-01 22:18:23 +02:00
70be78fd7b refactor: dedupe allanime requests 2026-06-01 22:17:02 +02:00
fbd2c5b602 refactor: dedupe watchlist ids 2026-06-01 22:15:21 +02:00
bfe23276ba refactor: dedupe proxy handlers 2026-06-01 22:14:15 +02:00
208281aee7 refactor: dedupe browse render 2026-06-01 22:12:49 +02:00
7943822194 refactor: dedupe allanime sources 2026-06-01 22:11:29 +02:00
d9ed4287a5 fix: hide scrollbar on studio and genre dropdowns 2026-06-01 19:04:03 +02:00
e907c7ae07 fix: hide episode list scrollbar on desktop 2026-06-01 19:00:11 +02:00
957905299e fix: give toggle inactive state a solid background 2026-06-01 18:58:55 +02:00
a865da79d4 fix: open More dropdown upward on watch page 2026-06-01 18:53:57 +02:00
156cb92fbe docs: add package comments to public and template packages 2026-06-01 12:55:53 +02:00
1861e20e2a docs: add package comments to server and watchlist packages 2026-06-01 12:55:48 +02:00
e146b0320a docs: add package comments to playback packages 2026-06-01 12:55:43 +02:00
fdd09bc004 docs: add package comments to anime and episodes packages 2026-06-01 12:55:38 +02:00
10c3923352 docs: add package comments to data layer packages 2026-06-01 12:55:33 +02:00
b862b6e08b docs: add package comments to auth and audit packages 2026-06-01 12:55:28 +02:00
5e553ceecc docs: add package comments to core infrastructure packages 2026-06-01 12:55:23 +02:00
475625de35 docs: add package comments to integrations 2026-06-01 12:55:20 +02:00
f4b3d1bccb feat: refacotr cmd/user/main.go 2026-05-31 19:17:05 +02:00
7f45e62dce refactor: extract generic graphql client 2026-05-31 19:02:35 +02:00
2b761127a0 chore: cleanup 2026-05-31 00:39:20 +02:00
1da19d500e feat: extract video module and add mode-switch fallback 2026-05-31 00:39:01 +02:00
2e3650b77b fix: sort scraped schedule entries by time within each day 2026-05-30 13:12:46 +02:00
9321f36a0f style: format cmd/readme table alignment 2026-05-29 21:24:27 +02:00
77acc627dc docs: improve readmes for cmd and template components 2026-05-29 21:24:00 +02:00
6929124ee3 fix: episode refresh resilience and allanime fallback 2026-05-29 21:12:53 +02:00
0b27974258 docs: clarify animeschedule api key is optional 2026-05-29 19:10:58 +02:00
dd38b1f7ba fix: remove forgejo ci/cd 2026-05-29 19:07:29 +02:00
c94e0699f3 feat: add create-user cli to image 2026-05-29 13:01:04 +02:00
cfb0ea724d feat: add end-state detection and prevent airing auto-complete 2026-05-29 00:04:17 +02:00
32586d6b08 feat: add airing status and end-state helpers to player 2026-05-29 00:04:05 +02:00
aebdd75942 fix: preserve watchlist progress on complete and status update 2026-05-29 00:03:47 +02:00
f89012f23c refactor: redesign schedule with responsive grid and expanded spacing 2026-05-28 23:24:50 +02:00
1242297742 feat: prefer english titles from animeschedule api 2026-05-28 23:24:39 +02:00
e8dcf1466b refactor: decompose anime handler and parallelize for-you fetches 2026-05-28 17:45:56 +02:00
54b03f85a2 refactor: remove CONFLICTS.md and inline avatar URL from migration 2026-05-28 12:52:10 +02:00
5dd49e585a refactor: extract CurrentUser and CurrentUserID helpers 2026-05-28 12:51:11 +02:00
04b241392c refactor: remove unused watchlist partial template 2026-05-28 12:47:38 +02:00
fd36b97908 refactor: replace regex parser with JSON walker in allanime extractor 2026-05-28 12:40:51 +02:00
f9a2649bec refactor: update template embed to remove anime subdirectory 2026-05-28 12:40:35 +02:00
dc09dcc547 refactor: update backfill migration to use internal.DefaultAvatarURL 2026-05-28 12:40:27 +02:00
271a24dbbe refactor: update user CLI to use internal.DefaultAvatarURL 2026-05-28 12:40:19 +02:00
363121465b refactor: update audit middleware to use flattened audit package 2026-05-28 12:40:11 +02:00
cf9c60ba70 refactor: update watchlist module imports for flattened package structure 2026-05-28 12:40:03 +02:00
cdebd407e4 refactor: update playback module imports for flattened package structure 2026-05-28 12:39:55 +02:00
82543d39fb refactor: update auth module imports for flattened package structure 2026-05-28 12:39:44 +02:00
c000e7c778 refactor: update audit module imports for flattened package structure 2026-05-28 12:39:35 +02:00
65a1d15383 refactor: update anime module imports for flattened package structure 2026-05-28 12:39:26 +02:00
7122a5d34d refactor: move reviews template from subdirectory 2026-05-28 12:39:07 +02:00
0b115e583d refactor: move watchlist service from subdirectory 2026-05-28 12:38:58 +02:00
589bf53597 refactor: move watchlist repository from subdirectory 2026-05-28 12:38:50 +02:00
6e2ba51c28 refactor: move watchlist handler from subdirectory 2026-05-28 12:38:42 +02:00
8b405845a1 refactor: move playback service from subdirectory 2026-05-28 12:38:34 +02:00
ceec637a43 refactor: move playback repository from subdirectory 2026-05-28 12:38:26 +02:00
21b84d7440 refactor: move anime service from subdirectory 2026-05-28 12:38:18 +02:00
7cdbcd7c04 refactor: move anime repository from subdirectory 2026-05-28 12:38:09 +02:00
68462d5591 refactor: move anime handler from subdirectory 2026-05-28 12:38:00 +02:00
9a0506913c refactor: move auth service from subdirectory 2026-05-28 12:37:51 +02:00
221155bed3 refactor: move auth repository from subdirectory 2026-05-28 12:37:42 +02:00
f9543d0d79 refactor: move auth middleware from subdirectory 2026-05-28 12:37:33 +02:00
d0d115cc93 refactor: move auth handler from subdirectory 2026-05-28 12:37:25 +02:00
f392610b4e refactor: move audit service test from internal/audit/service to internal/audit 2026-05-28 12:37:17 +02:00
3bbcc71460 refactor: move audit service from internal/audit/service to internal/audit 2026-05-28 12:37:09 +02:00
cb51800ae3 refactor: move audit context from internal/auditctx to internal/audit 2026-05-28 12:37:01 +02:00
b29fb5a3d6 refactor: move avatar from internal/users to internal 2026-05-28 12:36:50 +02:00
9695d7772d refactor: update playback handler imports for flattened pkg/net 2026-05-28 12:36:31 +02:00
866d293419 refactor: update watchorder imports for flattened pkg/net 2026-05-28 12:36:21 +02:00
91f6ba9db8 refactor: update allanime client imports for flattened pkg/net 2026-05-28 12:36:04 +02:00
3c1e4d34a9 refactor: update jikan imports for flattened pkg/net 2026-05-28 12:35:56 +02:00
186ea65545 refactor: update animeschedule imports for flattened pkg/net 2026-05-28 12:35:47 +02:00
3b930c5b79 refactor: flatten pkg/net/utls into pkg/net 2026-05-28 12:35:26 +02:00
0f96ec0c18 refactor: flatten pkg/net/useragent into pkg/net 2026-05-28 12:35:17 +02:00
0dbe4e75bc refactor: flatten pkg/net/proxytransport into pkg/net 2026-05-28 12:35:08 +02:00
05b9dfd216 refactor: flatten pkg/net/limits into pkg/net 2026-05-28 12:35:00 +02:00
f7e5f46234 refactor: move utls client from package var to provider field 2026-05-28 12:18:52 +02:00
fe0de5a214 refactor: centralize avatar URL generation and backfill existing users 2026-05-28 12:18:03 +02:00
dd4c7f80f3 feat: add transactional InTx to playback and watchlist repos 2026-05-28 12:17:19 +02:00
4329bce4a7 refactor: decouple domain types from jikan 2026-05-28 12:14:10 +02:00
6cc25af18a refactor: switch playback to AnimePlaybackService interface 2026-05-28 12:12:09 +02:00
3e67602e92 refactor: wire anime handler to use new service interfaces via fx 2026-05-28 12:12:00 +02:00
794eb8da27 refactor: split AnimeService into segregated interfaces 2026-05-28 12:11:53 +02:00
b52cd311a5 chore: format player main 2026-05-28 11:30:01 +02:00
a48d48f5ad chore: format player controls 2026-05-28 11:29:52 +02:00
8a21dadf21 chore: format player skip editor 2026-05-28 11:29:43 +02:00
a0c5005937 chore: format player skip index and segments 2026-05-28 11:29:33 +02:00
606df97eae chore: format player subtitles 2026-05-28 11:29:23 +02:00
fab242736d chore: format player episode nav and ui 2026-05-28 11:29:14 +02:00
15d311ace6 chore: format player episode complete and thumbnails 2026-05-28 11:29:04 +02:00
47b96107a5 chore: format player mode and state 2026-05-28 11:28:56 +02:00
2f88c14620 chore: format player progress quality keyboard 2026-05-28 11:28:46 +02:00
4a3e2e19d8 chore: format player storage and timeline 2026-05-28 11:28:36 +02:00
a4cf0375b7 chore: format watchlist 2026-05-28 11:28:25 +02:00
9f88e48786 chore: format search 2026-05-28 11:28:15 +02:00
8578bdb9e3 chore: format schedule_board 2026-05-28 11:28:02 +02:00
4b883c6572 chore: format toast and sort_filter 2026-05-28 11:27:52 +02:00
9375cf68b4 chore: format theme and timezone 2026-05-28 11:27:43 +02:00
c2931f941a chore: format htmx and shell 2026-05-28 11:27:32 +02:00
41128bd632 chore: format discover and dropdown 2026-05-28 11:27:22 +02:00
2823c6f026 chore: format anime and dedupe 2026-05-28 11:27:12 +02:00
fa7fe2f178 chore: format style.css 2026-05-28 11:27:01 +02:00
40204f04a1 chore: format small utility files 2026-05-28 11:26:51 +02:00
6868722061 chore: format scripts/new-data-fix.ts 2026-05-28 11:26:36 +02:00
049c78ac06 chore: update bun.lock for oxlint and oxfmt 2026-05-28 11:26:02 +02:00
eaeb2d09ee ci: replace prettier and eslint with oxfmt and oxlint 2026-05-28 11:25:36 +02:00
650a415c2d chore: replace eslint and prettier with oxlint and oxfmt 2026-05-28 11:25:26 +02:00
a9b20dff4c chore: update lefthook hooks for oxlint and oxfmt 2026-05-28 11:25:02 +02:00
28dc915a8d chore: remove eslint config 2026-05-28 11:24:53 +02:00
7b23d3f4c1 feat: add oxfmt configuration 2026-05-28 11:24:39 +02:00
843b98db5b feat: add oxlint configuration 2026-05-28 11:24:27 +02:00
3233894e6a ci: gracefully skip docker build if unavailable 2026-05-28 09:32:44 +02:00
dd482da9aa docs: remove ci section from readme 2026-05-28 09:11:47 +02:00
ef52daf3fa ci: use golangci-lint v2 install path 2026-05-28 09:08:38 +02:00
98e6ca64d1 ci: add forgejo actions workflows 2026-05-27 20:51:28 +02:00
4aa12e9fe5 chore: formatting 2026-05-27 14:05:35 +02:00
bb1eb8cb10 fix: pre push is no more 2026-05-27 12:08:52 +02:00
1076fa58b7 chore: formatting 2026-05-27 11:08:19 +02:00
69cfac8c9f fix: remove redundant type declaration 2026-05-27 11:03:11 +02:00
0ebe6e5963 docs: document ANIMESCHEDULE_API_TOKEN in readme 2026-05-27 11:02:16 +02:00
7e77f57a6f refine: adjust schedule board spacing and grid layout 2026-05-27 11:00:32 +02:00
ab37268e8b chore: remove debug logging from animeschedule integration 2026-05-27 11:00:25 +02:00
5dd6eedc3f feat: wire scraped schedule into handler with caching and week nav 2026-05-27 10:56:37 +02:00
c044ebdda0 feat: add schedule board client logic 2026-05-27 10:56:28 +02:00
c8e0c673ca feat: add animeschedule integration 2026-05-27 10:56:21 +02:00
150 changed files with 7982 additions and 5572 deletions

View File

@@ -3,15 +3,58 @@ version: "2"
linters: linters:
default: none default: none
enable: enable:
- bodyclose
- copyloopvar - copyloopvar
- cyclop
- dogsled
- dupl
- errcheck - errcheck
- funlen
- gocognit
- gocritic
- gocyclo
- govet - govet
- ineffassign - ineffassign
- maintidx
- makezero
- nakedret
- nilerr
- noctx
- prealloc
- predeclared
- revive - revive
- staticcheck - staticcheck
- unconvert - unconvert
- unparam
- unused - unused
- usestdlibvars
- wastedassign
- whitespace
settings: settings:
gocritic:
disable-all: true
enabled-checks:
- appendCombine
- boolExprSimplify
- commentedOutCode
- commentedOutImport
- deferUnlambda
- dupBranchBody
- dupImport
- dupSubExpr
- emptyDecl
- emptyFallthrough
- emptyStringTest
- equalFold
- redundantSprint
- regexpPattern
- stringConcatSimplify
- typeUnparen
- underef
- unlambda
- unnecessaryBlock
- unnecessaryDefer
- unslice
revive: revive:
enable-all-rules: false enable-all-rules: false
rules: rules:
@@ -39,7 +82,7 @@ linters:
- third_party$ - third_party$
- builtin$ - builtin$
- examples$ - examples$
- node_modules$ - node_modules/
issues: issues:
max-issues-per-linter: 0 max-issues-per-linter: 0

View File

@@ -18,9 +18,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN curl -fsSL https://bun.sh/install | bash RUN curl -fsSL https://bun.sh/install | bash
ENV PATH="/root/.bun/bin:${PATH}" ENV PATH="/root/.bun/bin:${PATH}"
# Install sqlc for code generation
RUN go install github.com/sqlc-dev/sqlc/cmd/sqlc@v1.30.0
ENV GOPROXY=direct ENV GOPROXY=direct
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
@@ -34,9 +31,6 @@ COPY . .
# Ensure dist is clean at build time (belt + suspenders) # Ensure dist is clean at build time (belt + suspenders)
RUN rm -rf dist/ && bun run build:assets RUN rm -rf dist/ && bun run build:assets
# Generate sqlc code
RUN sqlc generate
# Build the server and CLI tools # Build the server and CLI tools
RUN go build -ldflags="-s -w" -o main_server ./cmd/server RUN go build -ldflags="-s -w" -o main_server ./cmd/server
RUN go build -ldflags="-s -w" -o create-user ./cmd/user RUN go build -ldflags="-s -w" -o create-user ./cmd/user

View File

@@ -1,10 +1,7 @@
# MyAnimeList # MyAnimeList
<p align="center"> <p align="center">
<picture> <img src="/static/assets/logo.png" alt="MyAnimeList logo" width="120" />
<source media="(prefers-color-scheme: dark)" srcset="/static/assets/readme-logo-dark.svg" />
<img src="/static/assets/readme-logo-light.svg" alt="MyAnimeList logo" width="120" />
</picture>
</p> </p>
<p align="center"> <p align="center">
@@ -12,60 +9,47 @@
<img alt="SQLite" src="https://img.shields.io/badge/database-sqlite-003B57?style=flat-square&logo=sqlite" /> <img alt="SQLite" src="https://img.shields.io/badge/database-sqlite-003B57?style=flat-square&logo=sqlite" />
<img alt="Tailwind" src="https://img.shields.io/badge/tailwind-4-06D6D4?style=flat-square&logo=tailwindcss" /> <img alt="Tailwind" src="https://img.shields.io/badge/tailwind-4-06D6D4?style=flat-square&logo=tailwindcss" />
<img alt="HTMX" src="https://img.shields.io/badge/htmx-partial--updates-3366CC?style=flat-square" /> <img alt="HTMX" src="https://img.shields.io/badge/htmx-partial--updates-3366CC?style=flat-square" />
<img alt="License" src="https://img.shields.io/badge/license-MIT-green?style=flat-square" />
</p> </p>
--- MyAnimeList is a small self-hosted anime tracker and playback app. It keeps the catalog, watchlist, progress tracking, and player in one place, backed by a single SQLite database and a single Go server.
I built this because nothing else felt right. Every tracker I tried had decent pieces but the whole never clicked — awkward UI, missing features, or it just got in the way of actually watching anime. So I built one that fits how I work. Most of the UI is rendered on the server. HTMX handles lightweight updates like search, pagination, and watchlist changes, while TypeScript is kept for the parts that need real browser state: the video player, command palette, theme handling, and skip segment editor. The app also includes local users, API tokens, subtitle support, playlist rewriting, provider integrations, migrations, and startup data fixes.
It is a self-hosted Go server that streams anime through a proxy layer, catalogs metadata, and tracks your progress. ## Running
The frontend is Tailwind CSS v4 with HTMX handling pagination, infinite scroll, search, and watchlist interactions. TypeScript only steps in where HTMX cannot — the video player, command palette bound to Cmd+K, skip segment editor, theme toggling with system preference detection, and custom UI components. Everything lives in one process, one SQLite database, one deployment. Requires Go `1.25+`, Bun, [`just`](https://github.com/casey/just), and a C compiler for SQLite.
## Repository structure
| Path | Purpose |
| ----------------- | ------------------------------------------------ |
| `api/*` | Feature routes: anime, auth, playback, watchlist |
| `cmd/server` | Application entrypoint and CLI commands |
| `cmd/user` | User management CLI (create, update, delete) |
| `integrations/*` | External API clients and scraping |
| `internal/*` | Core services: db, middleware, server, worker |
| `pkg/middleware` | Generic HTTP middleware |
| `templates/*` | Server-rendered HTML templates |
| `migrations` | Schema evolution (20 migrations) |
| `static` / `dist` | Frontend assets |
## Running locally
Requires Go `1.25+`, Bun, and [just](https://github.com/casey/just). Migrations run on startup. Configuration lives in environment variables — see `cmd/server/main.go` for the full list.
An optional API key from [animeschedule.net](https://animeschedule.net) can be used for the schedule board to enable English titles and improve performance. Create an account, generate a token under your profile, and set it as `ANIMESCHEDULE_API_TOKEN`.
```bash ```bash
bun install
just build
go run ./cmd/user <username> <password>
just dev just dev
``` ```
## Quality checks The app starts on `http://localhost:3000` by default. Configuration comes from environment variables, and a local `.env` file is loaded automatically. The most useful options are `PORT`, `DATABASE_FILE`, `PLAYBACK_PROXY_SECRET`, `EPISODE_AVAILABILITY_MODE`, and `ANIMESCHEDULE_API_TOKEN`.
## Development
The codebase is split between Go feature packages, external integrations, server-rendered templates, and a small frontend asset pipeline. `cmd/server` starts the web app, `cmd/user` contains local admin tools, `internal` holds the application modules, `integrations` holds provider clients, and `templates`, `static`, and `dist` contain the UI.
The common development commands are in the `justfile`.
```bash ```bash
gofmt -l . just fmt
go test ./... just test
go build -o server ./cmd/server just lint-go
golangci-lint run ./... just lint-ts
go mod tidy just typecheck
go test -race ./... just build
bunx oxfmt --check
bun run lint:ts
bun run typecheck
bun run build:assets
docker build -t mal:ci .
``` ```
## Contributing Run the full local check with:
Bug reports and pull requests are welcome. This is a personal project, so there is no strict roadmap or issue triage cycle. If something is broken or missing, open an issue or send a PR. ```bash
just check
```
## License ## License
MIT. See `LICENSE`. MIT. See [`LICENSE`](LICENSE).

View File

@@ -5,6 +5,7 @@
"": { "": {
"name": "myanimelist-ui", "name": "myanimelist-ui",
"dependencies": { "dependencies": {
"hls.js": "^1.6.16",
"htmx.org": "1.9.12", "htmx.org": "1.9.12",
}, },
"devDependencies": { "devDependencies": {
@@ -185,6 +186,8 @@
"graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="], "graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="],
"hls.js": ["hls.js@1.6.16", "", {}, "sha512-VSIRpLfRwlAAdGL4wiTucx2ScRipo0ed1FBatWkyt832jC4CReKstga6yIhYVwGu9LOBjuX9wzmRMeQdBJtzEA=="],
"htmx.org": ["htmx.org@1.9.12", "", {}, "sha512-VZAohXyF7xPGS52IM8d1T1283y+X4D+Owf3qY1NZ9RuBypyu9l8cGsxUMAG5fEAb/DhT7rDoJ9Hpu5/HxFD3cw=="], "htmx.org": ["htmx.org@1.9.12", "", {}, "sha512-VZAohXyF7xPGS52IM8d1T1283y+X4D+Owf3qY1NZ9RuBypyu9l8cGsxUMAG5fEAb/DhT7rDoJ9Hpu5/HxFD3cw=="],
"is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="], "is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="],

View File

@@ -3,6 +3,7 @@ package main
import ( import (
"bufio" "bufio"
"context"
"database/sql" "database/sql"
"errors" "errors"
"fmt" "fmt"
@@ -36,6 +37,8 @@ func main() {
} }
func run(dbConn *sql.DB, args []string) int { func run(dbConn *sql.DB, args []string) int {
ctx := context.Background()
cmd, err := parseArgs(args) cmd, err := parseArgs(args)
if err != nil { if err != nil {
observability.Warn("cli_usage", "cmd/user", "invalid arguments", map[string]any{"argc": len(args)}, err) observability.Warn("cli_usage", "cmd/user", "invalid arguments", map[string]any{"argc": len(args)}, err)
@@ -45,13 +48,13 @@ func run(dbConn *sql.DB, args []string) int {
switch cmd.kind { switch cmd.kind {
case commandUpdateAvatar: case commandUpdateAvatar:
updateAvatars(dbConn) updateAvatars(ctx, dbConn)
return 0 return 0
case commandRunFixes: case commandRunFixes:
runFixes(dbConn) runFixes(ctx, dbConn)
return 0 return 0
case commandCreateOrUpdateUser: case commandCreateOrUpdateUser:
if err := createOrUpdateUser(dbConn, cmd.username, cmd.password); err != nil { if err := createOrUpdateUser(ctx, dbConn, cmd.username, cmd.password); err != nil {
return 1 return 1
} }
return 0 return 0
@@ -100,8 +103,8 @@ func usage() string {
return "Usage: go run cmd/user/main.go <username> <password>\n go run cmd/user/main.go update-avatar\n go run cmd/user/main.go run-fixes" return "Usage: go run cmd/user/main.go <username> <password>\n go run cmd/user/main.go update-avatar\n go run cmd/user/main.go run-fixes"
} }
func createOrUpdateUser(dbConn *sql.DB, username string, password string) error { func createOrUpdateUser(ctx context.Context, dbConn *sql.DB, username string, password string) error {
existingID, err := lookupUserID(dbConn, username) existingID, err := lookupUserID(ctx, dbConn, username)
if err != nil { if err != nil {
observability.Error("cli_user_lookup_failed", "cmd/user", "", map[string]any{"username": username}, err) observability.Error("cli_user_lookup_failed", "cmd/user", "", map[string]any{"username": username}, err)
return err return err
@@ -112,23 +115,23 @@ func createOrUpdateUser(dbConn *sql.DB, username string, password string) error
fmt.Println("Operation cancelled.") fmt.Println("Operation cancelled.")
return nil return nil
} }
if err := updateUserPassword(dbConn, existingID, username, password); err != nil { if err := updateUserPassword(ctx, dbConn, existingID, username, password); err != nil {
return err return err
} }
fmt.Printf("Password for '%s' updated successfully!\n", username) fmt.Printf("Password for '%s' updated successfully!\n", username)
return nil return nil
} }
if err := createUser(dbConn, username, password); err != nil { if err := createUser(ctx, dbConn, username, password); err != nil {
return err return err
} }
fmt.Printf("User '%s' was created successfully!\n", username) fmt.Printf("User '%s' was created successfully!\n", username)
return nil return nil
} }
func lookupUserID(dbConn *sql.DB, username string) (string, error) { func lookupUserID(ctx context.Context, dbConn *sql.DB, username string) (string, error) {
var id string var id string
err := dbConn.QueryRow("SELECT id FROM user WHERE username = ?", username).Scan(&id) err := dbConn.QueryRowContext(ctx, "SELECT id FROM user WHERE username = ?", username).Scan(&id)
if err == nil { if err == nil {
return id, nil return id, nil
} }
@@ -146,14 +149,14 @@ func promptConfirmOverwrite(username string) bool {
return response == "y" || response == "yes" return response == "y" || response == "yes"
} }
func updateUserPassword(dbConn *sql.DB, userID string, username string, password string) error { func updateUserPassword(ctx context.Context, dbConn *sql.DB, userID string, username string, password string) error {
hash, err := bcrypt.GenerateFromPassword([]byte(password), 12) hash, err := bcrypt.GenerateFromPassword([]byte(password), 12)
if err != nil { if err != nil {
observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err) observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err)
return err return err
} }
_, err = dbConn.Exec("UPDATE user SET password_hash = ? WHERE id = ?", string(hash), userID) _, err = dbConn.ExecContext(ctx, "UPDATE user SET password_hash = ? WHERE id = ?", string(hash), userID)
if err != nil { if err != nil {
observability.Error("cli_user_password_update_failed", "cmd/user", "", map[string]any{"username": username}, err) observability.Error("cli_user_password_update_failed", "cmd/user", "", map[string]any{"username": username}, err)
return err return err
@@ -161,7 +164,7 @@ func updateUserPassword(dbConn *sql.DB, userID string, username string, password
return nil return nil
} }
func createUser(dbConn *sql.DB, username string, password string) error { func createUser(ctx context.Context, dbConn *sql.DB, username string, password string) error {
hash, err := bcrypt.GenerateFromPassword([]byte(password), 12) hash, err := bcrypt.GenerateFromPassword([]byte(password), 12)
if err != nil { if err != nil {
observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err) observability.Error("cli_password_hash_failed", "cmd/user", "", nil, err)
@@ -170,7 +173,8 @@ func createUser(dbConn *sql.DB, username string, password string) error {
id := uuid.New().String() id := uuid.New().String()
avatarURL := internal.DefaultAvatarURL(username) avatarURL := internal.DefaultAvatarURL(username)
_, err = dbConn.Exec( _, err = dbConn.ExecContext(
ctx,
"INSERT INTO user (id, username, password_hash, avatar_url) VALUES (?, ?, ?, ?)", "INSERT INTO user (id, username, password_hash, avatar_url) VALUES (?, ?, ?, ?)",
id, id,
username, username,
@@ -184,8 +188,8 @@ func createUser(dbConn *sql.DB, username string, password string) error {
return nil return nil
} }
func updateAvatars(dbConn *sql.DB) { func updateAvatars(ctx context.Context, dbConn *sql.DB) {
rows, err := dbConn.Query("SELECT id, username FROM user") rows, err := dbConn.QueryContext(ctx, "SELECT id, username FROM user")
if err != nil { if err != nil {
observability.Error("cli_users_list_failed", "cmd/user", "", nil, err) observability.Error("cli_users_list_failed", "cmd/user", "", nil, err)
os.Exit(1) os.Exit(1)
@@ -201,7 +205,7 @@ func updateAvatars(dbConn *sql.DB) {
} }
avatarURL := internal.DefaultAvatarURL(username) avatarURL := internal.DefaultAvatarURL(username)
_, err := dbConn.Exec("UPDATE user SET avatar_url = ? WHERE id = ?", avatarURL, id) _, err := dbConn.ExecContext(ctx, "UPDATE user SET avatar_url = ? WHERE id = ?", avatarURL, id)
if err != nil { if err != nil {
observability.Error("cli_user_avatar_update_failed", "cmd/user", "", map[string]any{"username": username}, err) observability.Error("cli_user_avatar_update_failed", "cmd/user", "", map[string]any{"username": username}, err)
os.Exit(1) os.Exit(1)
@@ -217,13 +221,13 @@ func updateAvatars(dbConn *sql.DB) {
fmt.Printf("Updated avatars for %d user(s)\n", count) fmt.Printf("Updated avatars for %d user(s)\n", count)
} }
func runFixes(dbConn *sql.DB) { func runFixes(ctx context.Context, dbConn *sql.DB) {
if err := database.RunMigrationsAndFixes(dbConn); err != nil { if err := database.RunMigrationsAndFixes(dbConn); err != nil {
observability.Error("cli_run_migrations_and_fixes_failed", "cmd/user", "", nil, err) observability.Error("cli_run_migrations_and_fixes_failed", "cmd/user", "", nil, err)
os.Exit(1) os.Exit(1)
} }
rows, err := dbConn.Query("SELECT id, applied_at FROM data_fixes ORDER BY id ASC") rows, err := dbConn.QueryContext(ctx, "SELECT id, applied_at FROM data_fixes ORDER BY id ASC")
if err != nil { if err != nil {
observability.Error("cli_data_fixes_list_failed", "cmd/user", "", nil, err) observability.Error("cli_data_fixes_list_failed", "cmd/user", "", nil, err)
os.Exit(1) os.Exit(1)

View File

@@ -358,7 +358,7 @@ func fetchDocument(ctx context.Context, httpClient *http.Client, url string) (*g
return nil, url, err return nil, url, err
} }
return document, response.Request.URL.String(), nil return document, response, nil
} }
type timetableAnimeAPI struct { type timetableAnimeAPI struct {

View File

@@ -149,25 +149,43 @@ func jikanTraceEnabled() bool {
return traceEnabled return traceEnabled
} }
func logJikanCache(cacheKey string, source string, startedAt time.Time, err error) { func shouldSkipJikanCacheLog(source string, duration time.Duration, err error) bool {
duration := time.Since(startedAt) if jikanTraceEnabled() || err != nil {
if !jikanTraceEnabled() && err == nil && source == "fresh" && duration < 50*time.Millisecond { return false
return
}
if !jikanTraceEnabled() && err == nil && source == "refresh" && duration < jikanSlowLogThreshold {
return
} }
level := observability.LogLevelInfo if source == "fresh" {
return duration < 50*time.Millisecond
}
if source == "refresh" {
return duration < jikanSlowLogThreshold
}
return false
}
func jikanCacheLogLevel(source string, err error) observability.LogLevel {
if err != nil { if err != nil {
level = observability.LogLevelError return observability.LogLevelError
} else if source != "fresh" && source != "refresh" { }
if source != "fresh" && source != "refresh" {
// Stale reads are expected sometimes, but worth tracking in logs. // Stale reads are expected sometimes, but worth tracking in logs.
level = observability.LogLevelWarn return observability.LogLevelWarn
}
return observability.LogLevelInfo
}
func logJikanCache(cacheKey string, source string, startedAt time.Time, err error) {
duration := time.Since(startedAt)
if shouldSkipJikanCacheLog(source, duration, err) {
return
} }
observability.LogJSON( observability.LogJSON(
level, jikanCacheLogLevel(source, err),
"jikan_cache", "jikan_cache",
"jikan", "jikan",
"", "",
@@ -475,82 +493,116 @@ func (c *Client) fetchWithRetry(ctx context.Context, urlStr string, out any) err
for attempt := range maxRetries { for attempt := range maxRetries {
attempts = attempt + 1 attempts = attempt + 1
select { if err := c.prepareRetryAttempt(ctx); err != nil {
case <-ctx.Done():
return logAndReturn(0, fmt.Errorf("request canceled while retrying jikan request: %w", ctx.Err()))
default:
}
if err := c.waitRateLimit(ctx); err != nil {
return logAndReturn(0, err) return logAndReturn(0, err)
} }
req, err := http.NewRequestWithContext(ctx, http.MethodGet, urlStr, nil) resp, err := c.doRequest(ctx, urlStr)
if err != nil { if err != nil {
return logAndReturn(0, fmt.Errorf("failed to create jikan request: %w", err)) retry, requestErr := handleRequestRetry(ctx, err, attempt, maxRetries)
} if retry {
req.Header.Set("User-Agent", netutil.Generic)
resp, err := c.httpClient.Do(req)
if err != nil {
if errors.Is(err, context.Canceled) {
return logAndReturn(0, fmt.Errorf("request canceled while retrying jikan request: %w", err))
}
if attempt < maxRetries-1 && IsRetryableError(err) {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return logAndReturn(0, retryErr)
}
continue continue
} }
return logAndReturn(0, fmt.Errorf("jikan api error: %w", err)) return logAndReturn(0, requestErr)
} }
if resp.StatusCode != http.StatusOK { statusCode, retry, err := handleResponseRetry(ctx, resp, urlStr, out, attempt, maxRetries)
apiErr := &APIError{StatusCode: resp.StatusCode, URL: urlStr} if retry {
retryable := isRetryableStatus(resp.StatusCode)
retryAfter := time.Duration(0)
if parsed, ok := parseRetryAfter(resp.Header.Get("Retry-After")); ok {
retryAfter = parsed
}
if retryable && attempt < maxRetries-1 {
_ = resp.Body.Close()
delay := max(retryAfter, retryDelay(attempt))
if retryErr := waitForRetry(ctx, delay); retryErr != nil {
return logAndReturn(resp.StatusCode, retryErr)
}
continue
}
// Best-effort decode (often useful for debugging), but still treat non-200 as error.
_ = json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
return logAndReturn(resp.StatusCode, apiErr)
}
err = json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
if err == nil {
return logAndReturn(resp.StatusCode, nil)
}
if attempt < maxRetries-1 {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return logAndReturn(resp.StatusCode, retryErr)
}
continue continue
} }
return logAndReturn(resp.StatusCode, fmt.Errorf("failed to decode jikan response: %w", err)) return logAndReturn(statusCode, err)
} }
return logAndReturn(0, fmt.Errorf("max retries exceeded for %s", urlStr)) return logAndReturn(0, fmt.Errorf("max retries exceeded for %s", urlStr))
} }
func (c *Client) prepareRetryAttempt(ctx context.Context) error {
select {
case <-ctx.Done():
return fmt.Errorf("request canceled while retrying jikan request: %w", ctx.Err())
default:
}
return c.waitRateLimit(ctx)
}
func (c *Client) doRequest(ctx context.Context, urlStr string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, urlStr, nil)
if err != nil {
return nil, fmt.Errorf("failed to create jikan request: %w", err)
}
req.Header.Set("User-Agent", netutil.Generic)
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
return resp, nil
}
func handleRequestRetry(ctx context.Context, err error, attempt int, maxRetries int) (bool, error) {
if errors.Is(err, context.Canceled) {
return false, fmt.Errorf("request canceled while retrying jikan request: %w", err)
}
if attempt >= maxRetries-1 || !IsRetryableError(err) {
return false, fmt.Errorf("jikan api error: %w", err)
}
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return false, retryErr
}
return true, nil
}
func handleResponseRetry(ctx context.Context, resp *http.Response, urlStr string, out any, attempt int, maxRetries int) (int, bool, error) {
if resp.StatusCode != http.StatusOK {
return handleStatusRetry(ctx, resp, urlStr, out, attempt, maxRetries)
}
err := json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
if err == nil {
return resp.StatusCode, false, nil
}
if attempt < maxRetries-1 {
if retryErr := waitForRetry(ctx, retryDelay(attempt)); retryErr != nil {
return resp.StatusCode, false, retryErr
}
return resp.StatusCode, true, nil
}
return resp.StatusCode, false, fmt.Errorf("failed to decode jikan response: %w", err)
}
func handleStatusRetry(ctx context.Context, resp *http.Response, urlStr string, out any, attempt int, maxRetries int) (int, bool, error) {
statusCode := resp.StatusCode
apiErr := &APIError{StatusCode: statusCode, URL: urlStr}
retryAfter := time.Duration(0)
if parsed, ok := parseRetryAfter(resp.Header.Get("Retry-After")); ok {
retryAfter = parsed
}
if isRetryableStatus(statusCode) && attempt < maxRetries-1 {
_ = resp.Body.Close()
if retryErr := waitForRetry(ctx, max(retryAfter, retryDelay(attempt))); retryErr != nil {
return statusCode, false, retryErr
}
return statusCode, true, nil
}
// Best-effort decode (often useful for debugging), but still treat non-200 as error.
_ = json.NewDecoder(resp.Body).Decode(out)
_ = resp.Body.Close()
return statusCode, false, apiErr
}
func metricsEndpoint(urlStr string) string { func metricsEndpoint(urlStr string) string {
trimmed := strings.TrimSpace(urlStr) trimmed := strings.TrimSpace(urlStr)
if trimmed == "" { if trimmed == "" {

View File

@@ -23,42 +23,13 @@ func (fn roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
} }
func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) { func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
sqlDB, err := sql.Open("sqlite3", ":memory:") sqlDB := newTestCacheDB(t)
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
defer sqlDB.Close() defer sqlDB.Close()
sqlDB.SetMaxOpenConns(1)
_, err = sqlDB.Exec(`
CREATE TABLE jikan_cache (
key TEXT PRIMARY KEY,
data TEXT NOT NULL,
expires_at DATETIME NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
`)
if err != nil {
t.Fatalf("create cache table: %v", err)
}
queries := db.New(sqlDB) queries := db.New(sqlDB)
client := NewClient(config.Config{}, queries, observability.NewMetrics()) client := NewClient(config.Config{}, queries, observability.NewMetrics())
stale := TopAnimeResponse{Data: []Anime{{MalID: 1, Title: "stale"}}} stale := TopAnimeResponse{Data: []Anime{{MalID: 1, Title: "stale"}}}
staleBytes, err := json.Marshal(stale) insertCachedResponse(t, sqlDB, "top:1", stale, time.Now().Add(-time.Hour))
if err != nil {
t.Fatalf("marshal stale response: %v", err)
}
_, err = sqlDB.Exec(
`INSERT INTO jikan_cache (key, data, expires_at) VALUES (?, ?, ?)`,
"top:1",
string(staleBytes),
time.Now().Add(-time.Hour),
)
if err != nil {
t.Fatalf("insert stale cache: %v", err)
}
client.httpClient = &http.Client{ client.httpClient = &http.Client{
Transport: roundTripFunc(func(*http.Request) (*http.Response, error) { Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
@@ -78,11 +49,63 @@ func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
if len(got.Data) != 1 || got.Data[0].Title != "stale" { if len(got.Data) != 1 || got.Data[0].Title != "stale" {
t.Fatalf("got %+v, want stale cache response", got.Data) t.Fatalf("got %+v, want stale cache response", got.Data)
} }
waitForFreshCache(t, sqlDB, client, "top:1")
}
func newTestCacheDB(t *testing.T) *sql.DB {
t.Helper()
ctx := context.Background()
sqlDB, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
sqlDB.SetMaxOpenConns(1)
_, err = sqlDB.ExecContext(ctx, `
CREATE TABLE jikan_cache (
key TEXT PRIMARY KEY,
data TEXT NOT NULL,
expires_at DATETIME NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
`)
if err != nil {
sqlDB.Close()
t.Fatalf("create cache table: %v", err)
}
return sqlDB
}
func insertCachedResponse(t *testing.T, sqlDB *sql.DB, key string, value TopAnimeResponse, expiresAt time.Time) {
t.Helper()
ctx := context.Background()
encoded, err := json.Marshal(value)
if err != nil {
t.Fatalf("marshal cached response: %v", err)
}
_, err = sqlDB.ExecContext(
ctx,
`INSERT INTO jikan_cache (key, data, expires_at) VALUES (?, ?, ?)`,
key,
string(encoded),
expiresAt,
)
if err != nil {
t.Fatalf("insert cached response: %v", err)
}
}
func waitForFreshCache(t *testing.T, sqlDB *sql.DB, client *Client, key string) {
t.Helper()
deadline := time.Now().Add(2 * time.Second) deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) { for time.Now().Before(deadline) {
var refreshed TopAnimeResponse var refreshed TopAnimeResponse
if client.getCache(context.Background(), "top:1", &refreshed) && len(refreshed.Data) == 1 && refreshed.Data[0].Title == "fresh" { if client.getCache(context.Background(), key, &refreshed) && len(refreshed.Data) == 1 && refreshed.Data[0].Title == "fresh" {
return return
} }
time.Sleep(10 * time.Millisecond) time.Sleep(10 * time.Millisecond)
@@ -90,6 +113,6 @@ func TestGetWithCacheReturnsStaleAndRefreshesAsync(t *testing.T) {
var rawData string var rawData string
var rawExpires string var rawExpires string
_ = sqlDB.QueryRow(`SELECT data, expires_at FROM jikan_cache WHERE key = ?`, "top:1").Scan(&rawData, &rawExpires) _ = sqlDB.QueryRowContext(context.Background(), `SELECT data, expires_at FROM jikan_cache WHERE key = ?`, key).Scan(&rawData, &rawExpires)
t.Fatalf("cache was not refreshed asynchronously; data=%s expires_at=%s", rawData, rawExpires) t.Fatalf("cache was not refreshed asynchronously; data=%s expires_at=%s", rawData, rawExpires)
} }

View File

@@ -3,6 +3,8 @@ package jikan
import ( import (
"context" "context"
"fmt" "fmt"
"net/url"
"strconv"
"sync" "sync"
"time" "time"
) )
@@ -15,7 +17,9 @@ func (c *Client) GetEpisodes(ctx context.Context, animeID int, page int) (Episod
cacheKey := fmt.Sprintf("anime:%d:episodes:%d", animeID, page) cacheKey := fmt.Sprintf("anime:%d:episodes:%d", animeID, page)
var result EpisodesResponse var result EpisodesResponse
reqURL := fmt.Sprintf("%s/anime/%d/episodes?page=%d", c.baseURL, animeID, page) params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, fmt.Sprintf("/anime/%d/episodes", animeID), params)
err := c.getWithCache(ctx, cacheKey, 12*time.Hour, reqURL, &result) err := c.getWithCache(ctx, cacheKey, 12*time.Hour, reqURL, &result)
return result, err return result, err

View File

@@ -3,6 +3,8 @@ package jikan
import ( import (
"context" "context"
"fmt" "fmt"
"net/url"
"strconv"
) )
func (c *Client) GetAnimeStaff(ctx context.Context, id int) ([]StaffEntry, error) { func (c *Client) GetAnimeStaff(ctx context.Context, id int) ([]StaffEntry, error) {
@@ -46,7 +48,9 @@ func (c *Client) GetAnimeReviews(ctx context.Context, id int, page int) ([]Revie
page = 1 page = 1
} }
url := fmt.Sprintf("%s/anime/%d/reviews?page=%d", c.baseURL, id, page) params := url.Values{}
params.Set("page", strconv.Itoa(page))
url := buildRequestURL(c.baseURL, fmt.Sprintf("/anime/%d/reviews", id), params)
cacheKey := fmt.Sprintf("anime:reviews:%d:%d", id, page) cacheKey := fmt.Sprintf("anime:reviews:%d:%d", id, page)
var resp ReviewsResponse var resp ReviewsResponse

View File

@@ -56,10 +56,11 @@ func (c *Client) GetProducers(ctx context.Context, query string, page int, limit
func (c *Client) fetchProducersPage(ctx context.Context, query string, page int, limit int) (ProducerListResult, error) { func (c *Client) fetchProducersPage(ctx context.Context, query string, page int, limit int) (ProducerListResult, error) {
q := strings.TrimSpace(query) q := strings.TrimSpace(query)
cacheKey := fmt.Sprintf("producers:%s:%d:%d", q, page, limit) cacheKey := fmt.Sprintf("producers:%s:%d:%d", q, page, limit)
reqURL := fmt.Sprintf("%s/producers?page=%d&limit=%d", c.baseURL, page, limit) params := url.Values{}
if q != "" { params.Set("page", strconv.Itoa(page))
reqURL += "&q=" + url.QueryEscape(q) params.Set("limit", strconv.Itoa(limit))
} setQueryValue(params, "q", q)
reqURL := buildRequestURL(c.baseURL, "/producers", params)
var result ProducersResponse var result ProducersResponse
if err := c.getWithCache(ctx, cacheKey, producerCacheTTL, reqURL, &result); err != nil { if err := c.getWithCache(ctx, cacheKey, producerCacheTTL, reqURL, &result); err != nil {

View File

@@ -0,0 +1,43 @@
package jikan
import (
"fmt"
"net/url"
"strconv"
)
func buildRequestURL(baseURL, path string, params url.Values) string {
encoded := params.Encode()
if encoded == "" {
return fmt.Sprintf("%s%s", baseURL, path)
}
return fmt.Sprintf("%s%s?%s", baseURL, path, encoded)
}
func setQueryValue(values url.Values, key, value string) {
if value == "" {
values.Del(key)
return
}
values.Set(key, value)
}
func setPositiveIntQueryValue(values url.Values, key string, value int) {
if value <= 0 {
values.Del(key)
return
}
values.Set(key, strconv.Itoa(value))
}
func setTrueQueryValue(values url.Values, key string, enabled bool) {
if !enabled {
values.Del(key)
return
}
values.Set(key, "true")
}

View File

@@ -4,6 +4,7 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"net/http"
"sort" "sort"
"strings" "strings"
"time" "time"
@@ -20,6 +21,22 @@ const chiakiWatchOrderURL = "https://chiaki.site/?/tools/watch_order/id/%d"
const watchOrderCacheTTL = time.Hour * 24 const watchOrderCacheTTL = time.Hour * 24
const maxWatchOrderEntries = 120 // cap to prevent huge relation chains const maxWatchOrderEntries = 120 // cap to prevent huge relation chains
type WatchOrderMode string
const (
WatchOrderModeMain WatchOrderMode = "main"
WatchOrderModeComplete WatchOrderMode = "complete"
)
func NormalizeWatchOrderMode(value string) WatchOrderMode {
switch WatchOrderMode(strings.ToLower(strings.TrimSpace(value))) {
case WatchOrderModeComplete:
return WatchOrderModeComplete
default:
return WatchOrderModeMain
}
}
// watchOrderTypeLabel normalizes watch order type to display-friendly labels. // watchOrderTypeLabel normalizes watch order type to display-friendly labels.
func watchOrderTypeLabel(value string) string { func watchOrderTypeLabel(value string) string {
normalized := strings.ToLower(strings.TrimSpace(value)) normalized := strings.ToLower(strings.TrimSpace(value))
@@ -28,17 +45,35 @@ func watchOrderTypeLabel(value string) string {
return "TV" return "TV"
case "movie": case "movie":
return "Movie" return "Movie"
case "ona":
return "ONA"
case "ova":
return "OVA"
default: default:
return strings.TrimSpace(value) return strings.TrimSpace(value)
} }
} }
// isAllowedWatchOrderType returns true only for TV and Movie types (filters out specials, etc). func isTVWatchOrderType(value string) bool {
return strings.EqualFold(strings.TrimSpace(value), "tv")
}
// isAllowedWatchOrderType returns true for the default uncluttered watch order types.
func isAllowedWatchOrderType(value string) bool { func isAllowedWatchOrderType(value string) bool {
normalized := strings.ToLower(strings.TrimSpace(value)) normalized := strings.ToLower(strings.TrimSpace(value))
return normalized == "tv" || normalized == "movie" return normalized == "tv" || normalized == "movie"
} }
func hasTVWatchOrderEntry(entries []watchorder.WatchOrderEntry) bool {
for _, entry := range entries {
if isTVWatchOrderType(entry.Type) {
return true
}
}
return false
}
func relationCacheKey(id int) string { func relationCacheKey(id int) string {
return fmt.Sprintf("relations:watch-order:%d", id) return fmt.Sprintf("relations:watch-order:%d", id)
} }
@@ -52,7 +87,7 @@ func (c *Client) refreshWatchOrder(ctx context.Context, id int) (watchorder.Watc
result, err := watchorder.FetchWatchOrder(requestCtx, c.httpClient, watchOrderURL) result, err := watchorder.FetchWatchOrder(requestCtx, c.httpClient, watchOrderURL)
if err != nil { if err != nil {
var statusError *watchorder.HTTPStatusError var statusError *watchorder.HTTPStatusError
if errors.As(err, &statusError) && statusError.StatusCode == 404 { if errors.As(err, &statusError) && statusError.StatusCode == http.StatusNotFound {
return watchorder.WatchOrderResult{}, watchorder.ErrWatchOrderNotFound return watchorder.WatchOrderResult{}, watchorder.ErrWatchOrderNotFound
} }
if errors.Is(err, watchorder.ErrWatchOrderMarkupNotFound) { if errors.Is(err, watchorder.ErrWatchOrderMarkupNotFound) {
@@ -148,50 +183,54 @@ func (c *Client) currentOnlyRelation(ctx context.Context, id int) ([]RelationEnt
}}, nil }}, nil
} }
// GetFullRelations returns related anime based on watch order, with parallel fetching (3 concurrent). func (c *Client) handleWatchOrderError(ctx context.Context, id int, err error) ([]RelationEntry, error) {
func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry, error) { if errors.Is(err, watchorder.ErrWatchOrderNotFound) {
result, err := c.getWatchOrder(ctx, id)
if err != nil {
if errors.Is(err, watchorder.ErrWatchOrderNotFound) {
return c.currentOnlyRelation(ctx, id)
}
observability.Warn(
"relations_watch_order_fallback_current_only",
"jikan",
"",
map[string]any{
"anime_id": id,
},
err,
)
return c.currentOnlyRelation(ctx, id) return c.currentOnlyRelation(ctx, id)
} }
type fetchResult struct { observability.Warn(
index int "relations_watch_order_fallback_current_only",
anime Anime "jikan",
entry watchorder.WatchOrderEntry "",
} map[string]any{
"anime_id": id,
},
err,
)
var allowedEntries []watchorder.WatchOrderEntry return c.currentOnlyRelation(ctx, id)
}
func buildAllowedWatchOrderEntries(result watchorder.WatchOrderResult, mode WatchOrderMode) ([]watchorder.WatchOrderEntry, map[int]bool) {
allowedEntries := make([]watchorder.WatchOrderEntry, 0, len(result.WatchOrder))
seen := make(map[int]bool) seen := make(map[int]bool)
shouldIncludeAllTypes := mode == WatchOrderModeComplete || !hasTVWatchOrderEntry(result.WatchOrder)
for _, entry := range result.WatchOrder { for _, entry := range result.WatchOrder {
if len(allowedEntries) >= maxWatchOrderEntries { if len(allowedEntries) >= maxWatchOrderEntries {
break break
} }
if !isAllowedWatchOrderType(entry.Type) || seen[entry.ID] { if seen[entry.ID] {
continue continue
} }
if !shouldIncludeAllTypes && !isAllowedWatchOrderType(entry.Type) {
continue
}
seen[entry.ID] = true seen[entry.ID] = true
allowedEntries = append(allowedEntries, entry) allowedEntries = append(allowedEntries, entry)
} }
return allowedEntries, seen
}
func (c *Client) fetchRelationResults(ctx context.Context, entries []watchorder.WatchOrderEntry) []fetchResult {
g, gCtx := errgroup.WithContext(ctx) g, gCtx := errgroup.WithContext(ctx)
g.SetLimit(3) g.SetLimit(3)
results := make(chan fetchResult, len(allowedEntries)) results := make(chan fetchResult, len(entries))
for i, entry := range allowedEntries { for i, entry := range entries {
g.Go(func() error { g.Go(func() error {
anime, err := c.GetAnimeByID(gCtx, entry.ID) anime, err := c.GetAnimeByID(gCtx, entry.ID)
if err != nil { if err != nil {
@@ -201,10 +240,12 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
c.EnqueueAnimeFetchRetry(gCtx, entry.ID, err) c.EnqueueAnimeFetchRetry(gCtx, entry.ID, err)
return nil return nil
} }
select { select {
case results <- fetchResult{index: i, anime: anime, entry: entry}: case results <- fetchResult{index: i, anime: anime, entry: entry}:
case <-gCtx.Done(): case <-gCtx.Done():
} }
return nil return nil
}) })
} }
@@ -214,18 +255,21 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
close(results) close(results)
}() }()
fetched := make([]fetchResult, 0, len(allowedEntries)) fetched := make([]fetchResult, 0, len(entries))
for res := range results { for res := range results {
fetched = append(fetched, res) fetched = append(fetched, res)
} }
// Re-sort because they might have finished out of order
sort.Slice(fetched, func(i, j int) bool { sort.Slice(fetched, func(i, j int) bool {
return fetched[i].index < fetched[j].index return fetched[i].index < fetched[j].index
}) })
relations := make([]RelationEntry, 0, len(fetched)+1) return fetched
for _, res := range fetched { }
func buildRelationsFromResults(results []fetchResult, id int) []RelationEntry {
relations := make([]RelationEntry, 0, len(results)+1)
for _, res := range results {
relations = append(relations, RelationEntry{ relations = append(relations, RelationEntry{
Anime: res.anime, Anime: res.anime,
Relation: watchOrderTypeLabel(res.entry.Type), Relation: watchOrderTypeLabel(res.entry.Type),
@@ -234,18 +278,46 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
}) })
} }
if !seen[id] { return relations
currentAnime, err := c.GetAnimeByID(ctx, id) }
if err != nil {
return nil, err
}
relations = append([]RelationEntry{{ func (c *Client) ensureCurrentRelation(ctx context.Context, id int, seen map[int]bool, relations []RelationEntry) ([]RelationEntry, error) {
Anime: currentAnime, if seen[id] {
Relation: "Current", return relations, nil
IsCurrent: true, }
IsExtra: false,
}}, relations...) currentAnime, err := c.GetAnimeByID(ctx, id)
if err != nil {
return nil, err
}
return append([]RelationEntry{{
Anime: currentAnime,
Relation: "Current",
IsCurrent: true,
IsExtra: false,
}}, relations...), nil
}
type fetchResult struct {
index int
anime Anime
entry watchorder.WatchOrderEntry
}
// GetFullRelations returns related anime based on watch order, with parallel fetching (3 concurrent).
func (c *Client) GetFullRelations(ctx context.Context, id int, mode WatchOrderMode) ([]RelationEntry, error) {
result, err := c.getWatchOrder(ctx, id)
if err != nil {
return c.handleWatchOrderError(ctx, id, err)
}
allowedEntries, seen := buildAllowedWatchOrderEntries(result, mode)
fetched := c.fetchRelationResults(ctx, allowedEntries)
relations := buildRelationsFromResults(fetched, id)
relations, err = c.ensureCurrentRelation(ctx, id, seen, relations)
if err != nil {
return nil, err
} }
if len(relations) == 0 { if len(relations) == 0 {
@@ -257,6 +329,6 @@ func (c *Client) GetFullRelations(ctx context.Context, id int) ([]RelationEntry,
func (c *Client) WarmFullRelations(id int) { func (c *Client) WarmFullRelations(id int) {
c.runAsyncRefresh(func(ctx context.Context) { c.runAsyncRefresh(func(ctx context.Context) {
_, _ = c.GetFullRelations(ctx, id) _, _ = c.GetFullRelations(ctx, id, WatchOrderModeMain)
}) })
} }

View File

@@ -1,6 +1,9 @@
package jikan package jikan
import "testing" import (
"mal/integrations/watchorder"
"testing"
)
func runBoolCases(t *testing.T, tests []struct { func runBoolCases(t *testing.T, tests []struct {
name string name string
@@ -36,6 +39,138 @@ func TestIsAllowedWatchOrderType(t *testing.T) {
runBoolCases(t, tests, isAllowedWatchOrderType) runBoolCases(t, tests, isAllowedWatchOrderType)
} }
func TestNormalizeWatchOrderMode(t *testing.T) {
tests := []struct {
name string
input string
want WatchOrderMode
}{
{name: "empty defaults main", input: "", want: WatchOrderModeMain},
{name: "main", input: "main", want: WatchOrderModeMain},
{name: "complete", input: "complete", want: WatchOrderModeComplete},
{name: "case and whitespace", input: " COMPLETE ", want: WatchOrderModeComplete},
{name: "unknown defaults main", input: "everything", want: WatchOrderModeMain},
}
for _, testCase := range tests {
t.Run(testCase.name, func(t *testing.T) {
got := NormalizeWatchOrderMode(testCase.input)
if got != testCase.want {
t.Fatalf("expected %q, got %q", testCase.want, got)
}
})
}
}
func TestHasTVWatchOrderEntry(t *testing.T) {
tests := []struct {
name string
entries []watchorder.WatchOrderEntry
want bool
}{
{
name: "contains tv",
entries: []watchorder.WatchOrderEntry{
{ID: 1, Type: "Movie"},
{ID: 2, Type: " TV "},
},
want: true,
},
{
name: "ona only",
entries: []watchorder.WatchOrderEntry{
{ID: 1, Type: "ONA"},
{ID: 2, Type: "Special"},
},
want: false,
},
}
for _, testCase := range tests {
t.Run(testCase.name, func(t *testing.T) {
got := hasTVWatchOrderEntry(testCase.entries)
if got != testCase.want {
t.Fatalf("expected %v, got %v", testCase.want, got)
}
})
}
}
func TestBuildAllowedWatchOrderEntriesKeepsDefaultTypesWhenTVExists(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "TV"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "Movie"},
{ID: 4, Type: "ONA"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeMain)
if len(entries) != 2 {
t.Fatalf("expected 2 entries, got %d", len(entries))
}
if entries[0].ID != 1 || entries[1].ID != 3 {
t.Fatalf("unexpected entries: %+v", entries)
}
if !seen[1] || !seen[3] || seen[2] || seen[4] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestBuildAllowedWatchOrderEntriesIncludesAllTypesWhenNoTVExists(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "ONA"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "Movie"},
{ID: 1, Type: "ONA"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeMain)
if len(entries) != 3 {
t.Fatalf("expected 3 entries, got %d", len(entries))
}
if entries[0].ID != 1 || entries[1].ID != 2 || entries[2].ID != 3 {
t.Fatalf("unexpected entries: %+v", entries)
}
if !seen[1] || !seen[2] || !seen[3] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestBuildAllowedWatchOrderEntriesIncludesAllTypesInCompleteMode(t *testing.T) {
result := watchorder.WatchOrderResult{
WatchOrder: []watchorder.WatchOrderEntry{
{ID: 1, Type: "TV"},
{ID: 2, Type: "Special"},
{ID: 3, Type: "ONA"},
{ID: 4, Type: "Movie"},
},
}
entries, seen := buildAllowedWatchOrderEntries(result, WatchOrderModeComplete)
if len(entries) != 4 {
t.Fatalf("expected 4 entries, got %d", len(entries))
}
for index, entry := range entries {
wantID := index + 1
if entry.ID != wantID {
t.Fatalf("expected entry %d to have id %d, got %+v", index, wantID, entry)
}
}
if !seen[1] || !seen[2] || !seen[3] || !seen[4] {
t.Fatalf("unexpected seen map: %+v", seen)
}
}
func TestWatchOrderTypeLabel(t *testing.T) { func TestWatchOrderTypeLabel(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -44,6 +179,8 @@ func TestWatchOrderTypeLabel(t *testing.T) {
}{ }{
{name: "tv", input: "tv", want: "TV"}, {name: "tv", input: "tv", want: "TV"},
{name: "movie", input: "movie", want: "Movie"}, {name: "movie", input: "movie", want: "Movie"},
{name: "ona", input: "ona", want: "ONA"},
{name: "ova", input: "ova", want: "OVA"},
{name: "trimmed passthrough", input: " tv special ", want: "tv special"}, {name: "trimmed passthrough", input: " tv special ", want: "tv special"},
} }

View File

@@ -8,8 +8,7 @@ import (
"strings" "strings"
) )
// SearchAdvanced performs a filtered anime search with type, status, ordering, genre filters, and studio (producer) filters. func normalizeSearchPagination(page, limit int) (int, int) {
func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (SearchResult, error) {
if page < 1 { if page < 1 {
page = 1 page = 1
} }
@@ -17,46 +16,47 @@ func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, o
limit = 0 limit = 0
} }
genresParam := "" return page, limit
if len(genres) > 0 { }
ids := make([]string, len(genres))
for i, g := range genres { func joinGenreIDs(genres []int) string {
ids[i] = strconv.Itoa(g) if len(genres) == 0 {
} return ""
genresParam = strings.Join(ids, ",")
} }
ids := make([]string, len(genres))
for i, g := range genres {
ids[i] = strconv.Itoa(g)
}
return strings.Join(ids, ",")
}
func buildAdvancedSearchURL(baseURL, query, animeType, status, orderBy, sort, genres string, studioID int, sfw bool, page, limit int) string {
params := url.Values{}
params.Set("page", strconv.Itoa(page))
setTrueQueryValue(params, "sfw", sfw)
setQueryValue(params, "q", query)
setQueryValue(params, "type", animeType)
setQueryValue(params, "status", status)
setPositiveIntQueryValue(params, "producers", studioID)
setQueryValue(params, "order_by", orderBy)
setQueryValue(params, "sort", sort)
setQueryValue(params, "genres", genres)
setPositiveIntQueryValue(params, "limit", limit)
return buildRequestURL(baseURL, "/anime", params)
}
// SearchAdvanced performs a filtered anime search with type, status, ordering, genre filters, and studio (producer) filters.
func (c *Client) SearchAdvanced(ctx context.Context, query, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (SearchResult, error) {
page, limit = normalizeSearchPagination(page, limit)
genresParam := joinGenreIDs(genres)
cacheKey := fmt.Sprintf("search:%s:%s:%s:%s:%s:%s:%d:%v:%d:%d", query, animeType, status, orderBy, sort, genresParam, studioID, sfw, page, limit) cacheKey := fmt.Sprintf("search:%s:%s:%s:%s:%s:%s:%d:%v:%d:%d", query, animeType, status, orderBy, sort, genresParam, studioID, sfw, page, limit)
var result SearchResponse var result SearchResponse
reqURL := fmt.Sprintf("%s/anime?page=%d", c.baseURL, page) reqURL := buildAdvancedSearchURL(c.baseURL, query, animeType, status, orderBy, sort, genresParam, studioID, sfw, page, limit)
if sfw {
reqURL += "&sfw=true"
}
if query != "" {
reqURL += "&q=" + url.QueryEscape(query)
}
if animeType != "" {
reqURL += "&type=" + url.QueryEscape(animeType)
}
if status != "" {
reqURL += "&status=" + url.QueryEscape(status)
}
if studioID > 0 {
reqURL += "&producers=" + strconv.Itoa(studioID)
}
if orderBy != "" {
reqURL += "&order_by=" + url.QueryEscape(orderBy)
}
if sort != "" {
reqURL += "&sort=" + url.QueryEscape(sort)
}
if genresParam != "" {
reqURL += "&genres=" + genresParam
}
if limit > 0 {
reqURL += fmt.Sprintf("&limit=%d", limit)
}
if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil { if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil {
return SearchResult{}, err return SearchResult{}, err
@@ -76,7 +76,9 @@ func (c *Client) GetTopAnime(ctx context.Context, page int) (TopAnimeResult, err
cacheKey := fmt.Sprintf("top:%d", page) cacheKey := fmt.Sprintf("top:%d", page)
var result TopAnimeResponse var result TopAnimeResponse
reqURL := fmt.Sprintf("%s/top/anime?page=%d", c.baseURL, page) params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, "/top/anime", params)
if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil { if err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result); err != nil {
return TopAnimeResult{}, err return TopAnimeResult{}, err

View File

@@ -5,6 +5,8 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"math/rand" "math/rand"
"net/url"
"strconv"
"time" "time"
) )
@@ -30,7 +32,9 @@ func (c *Client) getSeasonList(ctx context.Context, page int, season string) (To
cacheKey := fmt.Sprintf("seasons_%s:%d", season, page) cacheKey := fmt.Sprintf("seasons_%s:%d", season, page)
var result TopAnimeResponse var result TopAnimeResponse
reqURL := fmt.Sprintf("%s/seasons/%s?page=%d", c.baseURL, season, page) params := url.Values{}
params.Set("page", strconv.Itoa(page))
reqURL := buildRequestURL(c.baseURL, fmt.Sprintf("/seasons/%s", season), params)
err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result) err := c.getWithCache(ctx, cacheKey, shortCacheTTL, reqURL, &result)
if err != nil { if err != nil {
@@ -44,76 +48,119 @@ func (c *Client) getSeasonList(ctx context.Context, page int, season string) (To
} }
// seedRandomPool seeds the in-memory pool of random anime // seedRandomPool seeds the in-memory pool of random anime
func (c *Client) seedRandomPool(ctx context.Context) error { func (c *Client) seedRandomPool(ctx context.Context) {
if !c.markRandomPoolInitialized() {
return
}
c.loadCachedRandomPool(ctx)
// Fetch a solid baseline in the background, then start refreshing.
go c.seedRandomPoolBaseline()
}
func (c *Client) markRandomPoolInitialized() bool {
c.poolMu.Lock() c.poolMu.Lock()
defer c.poolMu.Unlock()
if c.poolInitialized { if c.poolInitialized {
c.poolMu.Unlock() return false
}
c.poolInitialized = true
return true
}
func (c *Client) loadCachedRandomPool(ctx context.Context) {
cachedJSONs, err := c.db.GetAllCachedAnime(ctx)
if err != nil || len(cachedJSONs) == 0 {
return
}
loadedAnimes := decodeCachedAnime(cachedJSONs)
if len(loadedAnimes) == 0 {
return
}
c.poolMu.Lock()
c.randomPool = append(c.randomPool, loadedAnimes...)
c.poolMu.Unlock()
}
func decodeCachedAnime(cachedJSONs []string) []Anime {
loadedAnimes := make([]Anime, 0, len(cachedJSONs))
for _, dataStr := range cachedJSONs {
var anime Anime
if err := json.Unmarshal([]byte(dataStr), &anime); err != nil || anime.MalID == 0 {
continue
}
loadedAnimes = append(loadedAnimes, anime)
}
return loadedAnimes
}
func (c *Client) seedRandomPoolBaseline() {
bgCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
fetchedAnimes := c.fetchBaselineAnime(bgCtx)
if len(fetchedAnimes) > 0 {
c.appendUniqueRandomPool(fetchedAnimes)
}
// Start background refresher once seeding completes
c.startPoolRefresher()
}
func (c *Client) fetchBaselineAnime(ctx context.Context) []Anime {
topPageOne := c.fetchTopAnimePage(ctx, 1)
topPageTwo := c.fetchTopAnimePage(ctx, 2)
currentSeason := c.fetchCurrentSeasonAnime(ctx)
fetchedAnimes := make([]Anime, 0, len(topPageOne)+len(topPageTwo)+len(currentSeason))
fetchedAnimes = append(fetchedAnimes, topPageOne...)
fetchedAnimes = append(fetchedAnimes, topPageTwo...)
fetchedAnimes = append(fetchedAnimes, currentSeason...)
return fetchedAnimes
}
func (c *Client) fetchTopAnimePage(ctx context.Context, page int) []Anime {
top, err := c.GetTopAnime(ctx, page)
if err != nil {
return nil return nil
} }
c.poolInitialized = true
c.poolMu.Unlock()
// 1. Try to load all cached anime from the database return top.Animes
cachedJSONs, err := c.db.GetAllCachedAnime(ctx) }
if err == nil && len(cachedJSONs) > 0 {
var loadedAnimes []Anime
for _, dataStr := range cachedJSONs {
var anime Anime
if err := json.Unmarshal([]byte(dataStr), &anime); err == nil && anime.MalID > 0 {
loadedAnimes = append(loadedAnimes, anime)
}
}
if len(loadedAnimes) > 0 { func (c *Client) fetchCurrentSeasonAnime(ctx context.Context) []Anime {
c.poolMu.Lock() now, err := c.GetSeasonsNow(ctx, 1)
c.randomPool = append(c.randomPool, loadedAnimes...) if err != nil {
c.poolMu.Unlock() return nil
}
} }
// 2. Fetch Top Anime page 1 & 2 to ensure we have a robust baseline of high-quality popular anime return now.Animes
go func() { }
bgCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
var fetchedAnimes []Anime func (c *Client) appendUniqueRandomPool(animes []Anime) {
c.poolMu.Lock()
defer c.poolMu.Unlock()
top, err := c.GetTopAnime(bgCtx, 1) seen := make(map[int]bool, len(c.randomPool)+len(animes))
if err == nil && len(top.Animes) > 0 { for _, anime := range c.randomPool {
fetchedAnimes = append(fetchedAnimes, top.Animes...) seen[anime.MalID] = true
}
for _, anime := range animes {
if seen[anime.MalID] {
continue
} }
top2, err := c.GetTopAnime(bgCtx, 2) c.randomPool = append(c.randomPool, anime)
if err == nil && len(top2.Animes) > 0 { seen[anime.MalID] = true
fetchedAnimes = append(fetchedAnimes, top2.Animes...) }
}
now, err := c.GetSeasonsNow(bgCtx, 1)
if err == nil && len(now.Animes) > 0 {
fetchedAnimes = append(fetchedAnimes, now.Animes...)
}
if len(fetchedAnimes) > 0 {
c.poolMu.Lock()
// Use map to de-duplicate any anime
seen := make(map[int]bool)
for _, a := range c.randomPool {
seen[a.MalID] = true
}
for _, a := range fetchedAnimes {
if !seen[a.MalID] {
c.randomPool = append(c.randomPool, a)
seen[a.MalID] = true
}
}
c.poolMu.Unlock()
}
// Start background refresher once seeding completes
c.startPoolRefresher()
}()
return nil
} }
// startPoolRefresher runs in the background to slowly mix in true random anime // startPoolRefresher runs in the background to slowly mix in true random anime
@@ -162,7 +209,7 @@ func (c *Client) GetRandomAnime(ctx context.Context) (Anime, error) {
c.poolMu.Unlock() c.poolMu.Unlock()
if !initialized { if !initialized {
_ = c.seedRandomPool(ctx) c.seedRandomPool(ctx)
} }
c.poolMu.RLock() c.poolMu.RLock()

View File

@@ -33,12 +33,18 @@ type Aired struct {
String string `json:"string"` String string `json:"string"`
} }
type TitleEntry struct {
Type string `json:"type"`
Title string `json:"title"`
}
type Anime struct { type Anime struct {
MalID int `json:"mal_id"` MalID int `json:"mal_id"`
Title string `json:"title"` Title string `json:"title"`
TitleEnglish string `json:"title_english"` TitleEnglish string `json:"title_english"`
TitleJapanese string `json:"title_japanese"` TitleJapanese string `json:"title_japanese"`
TitleSynonyms []string `json:"title_synonyms"` TitleSynonyms []string `json:"title_synonyms"`
Titles []TitleEntry `json:"titles"`
Images struct { Images struct {
Jpg struct { Jpg struct {
LargeImageURL string `json:"large_image_url"` LargeImageURL string `json:"large_image_url"`
@@ -230,35 +236,34 @@ func (a Anime) DurationSeconds() float64 {
return 0 return 0
} }
var hours, minutes int var hours, minutes int
var isHours bool var currentValue int
var currentNum string hasValue := false
for _, c := range a.Duration { for _, token := range strings.Fields(strings.ToLower(a.Duration)) {
if c >= '0' && c <= '9' { value, err := strconv.Atoi(token)
currentNum += string(c) if err == nil {
} else if c == ' ' && currentNum != "" { currentValue = value
val, _ := strconv.Atoi(currentNum) hasValue = true
if isHours { continue
hours = val }
} else { if !hasValue {
minutes = val continue
} }
currentNum = ""
} else if len(currentNum) > 0 && (c == 'h' || c == 'H') { switch {
isHours = true case strings.HasPrefix(token, "h"):
val, _ := strconv.Atoi(currentNum) hours = currentValue
hours = val hasValue = false
currentNum = "" case strings.HasPrefix(token, "m"):
minutes = currentValue
hasValue = false
} }
} }
if currentNum != "" {
val, _ := strconv.Atoi(currentNum) if hasValue {
if isHours { minutes = currentValue
hours = val
} else {
minutes = val
}
} }
return float64(hours*60+minutes) * 60 return float64(hours*60+minutes) * 60
} }
@@ -455,13 +460,16 @@ type ReviewsResponse struct {
Pagination Pagination `json:"pagination"` Pagination Pagination `json:"pagination"`
} }
// DisplayTitle returns English title if available, otherwise Japanese, then default. // DisplayTitle returns English title if available, otherwise default title, titles[0], then Japanese.
func (a Anime) DisplayTitle() string { func (a Anime) DisplayTitle() string {
if a.TitleEnglish != "" { if a.TitleEnglish != "" {
return a.TitleEnglish return a.TitleEnglish
} }
if a.TitleJapanese != "" { if a.Title != "" {
return a.TitleJapanese return a.Title
} }
return a.Title if len(a.Titles) > 0 && a.Titles[0].Title != "" {
return a.Titles[0].Title
}
return a.TitleJapanese
} }

View File

@@ -0,0 +1,27 @@
package jikan
import "testing"
func TestAnimeDisplayTitlePrefersTitleBeforeJapanese(t *testing.T) {
anime := Anime{
Title: "Cyberpunk: Edgerunners",
TitleJapanese: "サイバーパンク エッジランナーズ",
}
if got := anime.DisplayTitle(); got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want default title", got)
}
}
func TestAnimeDisplayTitleFallsBackToFirstTitleEntryBeforeJapanese(t *testing.T) {
anime := Anime{
TitleJapanese: "サイバーパンク エッジランナーズ",
Titles: []TitleEntry{
{Type: "Default", Title: "Cyberpunk: Edgerunners"},
},
}
if got := anime.DisplayTitle(); got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want first title entry", got)
}
}

View File

@@ -0,0 +1,102 @@
package allanime
import (
"context"
"fmt"
"mal/internal/domain"
"strconv"
"strings"
)
type AvailableEpisodes struct {
Sub []string
Dub []string
Raw []string
}
func (c *AllAnimeProvider) GetEpisodeAvailability(ctx context.Context, animeID int, titleCandidates []string) (domain.EpisodeAvailability, error) {
showID, err := c.ResolveEpisodeProviderID(ctx, animeID, titleCandidates)
if err != nil {
return domain.EpisodeAvailability{}, err
}
return c.GetEpisodeAvailabilityByProviderID(ctx, showID)
}
func (c *AllAnimeProvider) GetEpisodeAvailabilityByProviderID(ctx context.Context, showID string) (domain.EpisodeAvailability, error) {
available, err := c.GetAvailableEpisodes(ctx, showID)
if err != nil {
return domain.EpisodeAvailability{}, err
}
sub := parseEpisodeNumbers(append(available.Sub, available.Raw...))
dub := parseEpisodeNumbers(available.Dub)
return domain.EpisodeAvailability{Sub: sub, Dub: dub}, nil
}
func (c *AllAnimeProvider) GetAvailableEpisodes(ctx context.Context, showID string) (AvailableEpisodes, error) {
graphqlQuery := `query($showId: String!) {
show(_id: $showId) {
availableEpisodesDetail
lastEpisodeInfo
}
}`
result, err := c.graphqlRequest(ctx, graphqlQuery, map[string]any{"showId": showID})
if err != nil {
return AvailableEpisodes{}, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid response")
}
show, ok := data["show"].(map[string]any)
if !ok || show == nil {
return AvailableEpisodes{}, fmt.Errorf("show not found")
}
detail, ok := show["availableEpisodesDetail"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid detail")
}
return AvailableEpisodes{
Sub: stringSliceFromAny(detail["sub"]),
Dub: stringSliceFromAny(detail["dub"]),
Raw: stringSliceFromAny(detail["raw"]),
}, nil
}
func parseEpisodeNumbers(raw []string) []int {
seen := make(map[int]bool, len(raw))
out := make([]int, 0, len(raw))
for _, value := range raw {
n, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil || n <= 0 || seen[n] {
continue
}
seen[n] = true
out = append(out, n)
}
return out
}
func stringSliceFromAny(value any) []string {
items, ok := value.([]any)
if !ok {
return nil
}
values := make([]string, 0, len(items))
for _, item := range items {
str, ok := item.(string)
if !ok {
continue
}
values = append(values, str)
}
return values
}

View File

@@ -1,49 +1,26 @@
// Package allanime provides an integration with the AllAnime API for episode playback.
package allanime package allanime
import ( import (
"bytes" "bytes"
"context" "context"
"crypto/aes"
"crypto/cipher"
"crypto/sha256"
"encoding/base64"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
"mal/internal/domain" "mal/internal/domain"
"mal/pkg"
netutil "mal/pkg/net" netutil "mal/pkg/net"
"net/http" "net/http"
"net/url"
"strconv"
"strings" "strings"
"time" "time"
) )
const ( const (
allAnimeBaseURL = "https://api.allanime.day" allAnimeBaseURL = "https://api.allanime.day"
allAnimeReferer = "https://allmanga.to/" allAnimeSiteURL = "https://allanime.day"
allAnimeReferer = "https://youtu-chan.com"
allAnimeOrigin = "https://youtu-chan.com" allAnimeOrigin = "https://youtu-chan.com"
defaultUserAgent = netutil.Firefox121 defaultUserAgent = netutil.Firefox121
) )
var (
aesKeys = []string{"Xot36i3lK3:v1", "SimtVuagFbGR2K7P"}
)
type searchResult struct {
ID string
MalID string
Name string
}
type AvailableEpisodes struct {
Sub []string
Dub []string
Raw []string
}
type AllAnimeProvider struct { type AllAnimeProvider struct {
httpClient *http.Client httpClient *http.Client
utlsClient *http.Client utlsClient *http.Client
@@ -67,139 +44,23 @@ func (c *AllAnimeProvider) Name() string {
return "AllAnime" return "AllAnime"
} }
const searchQuery = `query(
$search: SearchInput
$translationType: VaildTranslationTypeEnumType
$limit: Int = 40
$page: Int = 1
$countryOrigin: VaildCountryOriginEnumType = ALL
) {
shows(
search: $search
limit: $limit
page: $page
translationType: $translationType
countryOrigin: $countryOrigin
) {
edges {
_id
malId
name
}
}
}`
func (c *AllAnimeProvider) Search(ctx context.Context, query string, mode string) ([]searchResult, error) {
type searchData struct {
Shows struct {
Edges []struct {
ID string `json:"_id"`
MalID string `json:"malId"`
Name string `json:"name"`
} `json:"edges"`
} `json:"shows"`
}
type searchInput struct {
AllowAdult bool `json:"allowAdult"`
AllowUnknown bool `json:"allowUnknown"`
Query string `json:"query"`
}
type searchVariables struct {
Search searchInput `json:"search"`
TranslationType string `json:"translationType"`
}
vars := searchVariables{
Search: searchInput{
AllowAdult: false,
AllowUnknown: false,
Query: query,
},
TranslationType: mode,
}
data, err := graphql.Post[searchData](ctx, c.httpClient, allAnimeBaseURL+"/api", searchQuery, vars, graphql.PostOptions{
Headers: map[string]string{
"Referer": allAnimeReferer,
"User-Agent": defaultUserAgent,
},
BodyMax: netutil.MiB2,
})
if err != nil {
return nil, err
}
out := make([]searchResult, 0, len(data.Shows.Edges))
for _, edge := range data.Shows.Edges {
id := edge.ID
malID := edge.MalID
name := edge.Name
if unquoted, err := strconv.Unquote("\"" + name + "\""); err == nil {
name = unquoted
}
name = strings.TrimSpace(name)
if id == "" {
continue
}
out = append(out, searchResult{ID: id, MalID: malID, Name: name})
}
return out, nil
}
func (c *AllAnimeProvider) GetStreams(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string) (*domain.StreamResult, error) { func (c *AllAnimeProvider) GetStreams(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string) (*domain.StreamResult, error) {
// 1. Search for the show to get its AllAnime ID showID := c.resolveShowIDWithFallback(ctx, animeID, titleCandidates, mode)
// Try each title candidate, preferring results with matching malId
targetMalIDStr := strconv.Itoa(animeID)
var showID string
var firstAvailableShowID string
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil || len(searchResults) == 0 {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
showID = res.ID
break
}
}
if showID != "" {
break
}
if firstAvailableShowID == "" {
firstAvailableShowID = searchResults[0].ID
}
}
if showID == "" {
showID = firstAvailableShowID
}
if showID == "" { if showID == "" {
return nil, fmt.Errorf("allanime: show not found for malID %d", animeID) return nil, fmt.Errorf("allanime: show not found for malID %d", animeID)
} }
// 2. Get sources
sources, err := c.GetEpisodeSources(ctx, showID, episode, mode) sources, err := c.GetEpisodeSources(ctx, showID, episode, mode)
if err != nil || len(sources) == 0 { if err != nil || len(sources) == 0 {
return nil, fmt.Errorf("allanime: no sources for show %s", showID) return nil, fmt.Errorf("allanime: no sources for show %s", showID)
} }
// 3. Return the first usable source
primary := sources[0] primary := sources[0]
result := &domain.StreamResult{ result := &domain.StreamResult{
URL: primary.URL, URL: primary.URL,
Referer: primary.Referer, Referer: primary.Referer,
Type: primary.Type,
} }
for _, sub := range primary.Subtitles { for _, sub := range primary.Subtitles {
@@ -212,65 +73,6 @@ func (c *AllAnimeProvider) GetStreams(ctx context.Context, animeID int, titleCan
return result, nil return result, nil
} }
func (c *AllAnimeProvider) GetEpisodeAvailability(ctx context.Context, animeID int, titleCandidates []string) (domain.EpisodeAvailability, error) {
showID, err := c.ResolveEpisodeProviderID(ctx, animeID, titleCandidates)
if err != nil {
return domain.EpisodeAvailability{}, err
}
return c.GetEpisodeAvailabilityByProviderID(ctx, showID)
}
func (c *AllAnimeProvider) ResolveEpisodeProviderID(ctx context.Context, animeID int, titleCandidates []string) (string, error) {
for _, mode := range []string{"sub", "dub"} {
showID, err := c.resolveShowIDStrict(ctx, animeID, titleCandidates, mode)
if err == nil {
return showID, nil
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d", animeID)
}
func (c *AllAnimeProvider) GetEpisodeAvailabilityByProviderID(ctx context.Context, showID string) (domain.EpisodeAvailability, error) {
available, err := c.GetAvailableEpisodes(ctx, showID)
if err != nil {
return domain.EpisodeAvailability{}, err
}
sub := parseEpisodeNumbers(append(available.Sub, available.Raw...))
dub := parseEpisodeNumbers(available.Dub)
return domain.EpisodeAvailability{Sub: sub, Dub: dub}, nil
}
func (c *AllAnimeProvider) resolveShowIDStrict(ctx context.Context, animeID int, titleCandidates []string, mode string) (string, error) {
targetMalIDStr := strconv.Itoa(animeID)
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
return res.ID, nil
}
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d in %s search", animeID, mode)
}
func parseEpisodeNumbers(raw []string) []int {
seen := make(map[int]bool, len(raw))
out := make([]int, 0, len(raw))
for _, value := range raw {
n, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil || n <= 0 || seen[n] {
continue
}
seen[n] = true
out = append(out, n)
}
return out
}
func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, variables map[string]any) (map[string]any, error) { func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, variables map[string]any) (map[string]any, error) {
if mode, ok := variables["translationType"].(string); ok { if mode, ok := variables["translationType"].(string); ok {
variables["translationType"] = strings.ToLower(mode) variables["translationType"] = strings.ToLower(mode)
@@ -295,13 +97,13 @@ func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, var
req.Header.Set("Referer", allAnimeReferer) req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("User-Agent", defaultUserAgent) req.Header.Set("User-Agent", defaultUserAgent)
resp, respBody, err := executeAndReadResponse(c.httpClient, req, "execute graphql request", "read graphql response") statusCode, respBody, err := executeAndReadResponse(c.httpClient, req, "execute graphql request", "read graphql response")
if err != nil { if err != nil {
return nil, err return nil, err
} }
if resp.StatusCode != http.StatusOK { if statusCode != http.StatusOK {
return nil, fmt.Errorf("graphql status %d", resp.StatusCode) return nil, fmt.Errorf("graphql status %d", statusCode)
} }
var parsed map[string]any var parsed map[string]any
@@ -316,487 +118,17 @@ func (c *AllAnimeProvider) graphqlRequest(ctx context.Context, query string, var
return parsed, nil return parsed, nil
} }
const episodeQueryHash = "d405d0edd690624b66baba3068e0edc3ac90f1597d898a1ec8db4e5c43c00fec" func executeAndReadResponse(client *http.Client, req *http.Request, executeErrPrefix string, readErrPrefix string) (int, []byte, error) {
func (c *AllAnimeProvider) graphqlRequestWithHash(ctx context.Context, showID, episode, mode string) (map[string]any, error) {
mode = strings.ToLower(mode)
varsJSON := fmt.Sprintf(`{"showId":"%s","translationType":"%s","episodeString":"%s"}`, showID, mode, episode)
extJSON := fmt.Sprintf(`{"persistedQuery":{"version":1,"sha256Hash":"%s"}}`, episodeQueryHash)
apiURL := fmt.Sprintf("%s/api?variables=%s&extensions=%s",
allAnimeBaseURL,
url.QueryEscape(varsJSON),
url.QueryEscape(extJSON))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, apiURL, nil)
if err != nil {
return nil, fmt.Errorf("create GET request: %w", err)
}
req.Header.Set("User-Agent", defaultUserAgent)
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Language", "en-US,en;q=0.5")
req.Header.Set("Accept-Encoding", "identity")
req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("Origin", allAnimeOrigin)
req.Header.Set("Sec-Fetch-Dest", "empty")
req.Header.Set("Sec-Fetch-Mode", "cors")
req.Header.Set("Sec-Fetch-Site", "cross-site")
resp, respBody, err := executeAndReadResponse(c.utlsClient, req, "execute GET request", "read response")
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("GET status %d: %s", resp.StatusCode, string(respBody))
}
var parsed map[string]any
if err := json.Unmarshal(respBody, &parsed); err != nil {
return nil, fmt.Errorf("decode response: %w", err)
}
if errs, ok := parsed["errors"].([]any); ok && len(errs) > 0 {
return nil, fmt.Errorf("graphql error: %v", errs[0])
}
data, ok := parsed["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("no data in response")
}
var toBeParsed string
if s, ok := data["tobeparsed"].(string); ok && s != "" {
toBeParsed = s
} else if episodeData, ok := data["episode"].(map[string]any); ok {
if s, ok := episodeData["tobeparsed"].(string); ok {
toBeParsed = s
}
}
if toBeParsed != "" {
decrypted, err := decryptTobeparsed(toBeParsed)
if err != nil {
return nil, fmt.Errorf("decrypt tobeparsed: %w", err)
}
var ep map[string]any
if jerr := json.Unmarshal(decrypted, &ep); jerr != nil {
return nil, fmt.Errorf("unmarshal decrypted: %w", jerr)
}
var sourceURLs []any
if srcs, ok := ep["sourceUrls"].([]any); ok {
sourceURLs = srcs
} else if epInner, ok := ep["episode"].(map[string]any); ok {
if srcs, ok := epInner["sourceUrls"].([]any); ok {
sourceURLs = srcs
}
}
if len(sourceURLs) > 0 {
return map[string]any{
"episode": map[string]any{
"sourceUrls": sourceURLs,
},
}, nil
}
}
if episodeData, ok := data["episode"].(map[string]any); ok {
if srcs, ok := episodeData["sourceUrls"].([]any); ok && len(srcs) > 0 {
return parsed, nil
}
}
return nil, fmt.Errorf("no usable data in response")
}
// GetEpisodeSources fetches stream URLs for a given show, episode, and mode (dub/sub).
func (c *AllAnimeProvider) GetEpisodeSources(ctx context.Context, showID string, episode string, mode string) ([]StreamSource, error) {
episodeQuery := `query($showId: String!, $translationType: VaildTranslationTypeEnumType!, $episodeString: String!) {
episode(showId: $showId, translationType: $translationType, episodeString: $episodeString) {
sourceUrls
}
}`
result, err := c.graphqlRequestWithHash(ctx, showID, episode, mode)
if err == nil {
sources := c.extractSourceURLsFromData(ctx, result)
if len(sources) > 0 {
return sources, nil
}
}
result, err = c.graphqlRequest(ctx, episodeQuery, map[string]any{
"showId": showID,
"translationType": mode,
"episodeString": episode,
})
if err != nil {
return nil, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid source response")
}
rawSourceURLs, ok := data["episode"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid episode response")
}
sourceURLs, ok := rawSourceURLs["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil, fmt.Errorf("no source urls")
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil, fmt.Errorf("no source references")
}
out := c.resolveSourceReferences(ctx, references)
if len(out) == 0 {
return nil, fmt.Errorf("no playable sources extracted")
}
return out, nil
}
func (c *AllAnimeProvider) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource {
episodeData, ok := data["episode"].(map[string]any)
if !ok {
return nil
}
sourceURLs, ok := episodeData["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil
}
return c.resolveSourceReferences(ctx, references)
}
func (c *AllAnimeProvider) resolveSourceReferences(ctx context.Context, references []sourceReference) []StreamSource {
out := make([]StreamSource, 0, len(references))
for _, ref := range references {
target := strings.TrimSpace(ref.URL)
if target == "" {
continue
}
if strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") {
sourceType := detectStreamType(target)
if sourceType == "unknown" {
sourceType = detectEmbedType(target)
}
out = append(out, buildStreamSource(target, sourceType, ref.Name))
continue
}
decoded := decodeSourceURL(target)
if decoded == "" {
continue
}
if strings.HasPrefix(decoded, "http://") || strings.HasPrefix(decoded, "https://") {
sourceType := detectStreamType(decoded)
if sourceType == "unknown" {
sourceType = detectEmbedType(decoded)
}
out = append(out, buildStreamSource(decoded, sourceType, ref.Name))
continue
}
if !strings.HasPrefix(decoded, "/") {
decoded = "/" + decoded
}
extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded)
if err != nil {
continue
}
out = append(out, extracted...)
}
return out
}
func executeAndReadResponse(client *http.Client, req *http.Request, executeErrPrefix string, readErrPrefix string) (*http.Response, []byte, error) {
resp, err := client.Do(req) resp, err := client.Do(req)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("%s: %w", executeErrPrefix, err) return 0, nil, fmt.Errorf("%s: %w", executeErrPrefix, err)
} }
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2)) body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("%s: %w", readErrPrefix, err) return 0, nil, fmt.Errorf("%s: %w", readErrPrefix, err)
} }
return resp, body, nil return resp.StatusCode, body, nil
}
func buildStreamSource(url, sourceType, provider string) StreamSource {
return StreamSource{
URL: url,
Provider: provider,
Type: sourceType,
Referer: allAnimeReferer,
}
}
type sourceReference struct {
URL string
Name string
}
// buildSourceReferences orders source URLs by provider priority, deduplicating entries.
func buildSourceReferences(rawSourceURLs []any) []sourceReference {
priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"}
prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}}
prioritized := make(map[string]sourceReference)
fallback := make([]sourceReference, 0, len(rawSourceURLs))
seen := make(map[string]struct{})
for _, source := range rawSourceURLs {
item, ok := source.(map[string]any)
if !ok {
continue
}
sourceURL, _ := item["sourceUrl"].(string)
sourceName, _ := item["sourceName"].(string)
sourceURL = strings.TrimSpace(sourceURL)
sourceName = strings.TrimSpace(sourceName)
if sourceURL == "" {
continue
}
if _, exists := seen[sourceURL]; exists {
continue
}
seen[sourceURL] = struct{}{}
ref := sourceReference{URL: sourceURL, Name: sourceName}
normalized := strings.ToLower(sourceName)
// separate prioritized providers from fallback
if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider {
if _, exists := prioritized[normalized]; !exists {
prioritized[normalized] = ref
}
continue
}
fallback = append(fallback, ref)
}
// output: prioritized in order, then fallback
ordered := make([]sourceReference, 0, len(prioritized)+len(fallback))
for _, provider := range priorityOrder {
if ref, ok := prioritized[provider]; ok {
ordered = append(ordered, ref)
}
}
ordered = append(ordered, fallback...)
return ordered
}
func decryptTobeparsed(encoded string) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
return nil, fmt.Errorf("base64 decode failed: %w", err)
}
if len(raw) < 29 {
return nil, fmt.Errorf("encrypted payload too short")
}
version := raw[0]
iv := raw[1:13]
cipherText := raw[13 : len(raw)-16]
for _, keyStr := range aesKeys {
key := sha256.Sum256([]byte(keyStr))
block, err := aes.NewCipher(key[:])
if err != nil {
continue
}
if version == 1 {
plainText := tryDecryptCTR(block, iv, cipherText)
if json.Valid(plainText) {
return plainText, nil
}
}
gcm, err := cipher.NewGCM(block)
if err == nil {
tag := raw[len(raw)-16:]
combined := append(append([]byte{}, cipherText...), tag...)
plainText, openErr := gcm.Open(nil, iv, combined, nil)
if openErr == nil && json.Valid(plainText) {
return plainText, nil
}
}
}
return nil, fmt.Errorf("decryption failed")
}
func tryDecryptCTR(block cipher.Block, iv []byte, cipherText []byte) []byte {
ctrIV := append([]byte{}, iv...)
ctrIV = append(ctrIV, 0x00, 0x00, 0x00, 0x02)
ctr := cipher.NewCTR(block, ctrIV)
plainText := make([]byte, len(cipherText))
ctr.XORKeyStream(plainText, cipherText)
return plainText
}
// GetAvailableEpisodes returns the count of sub/dub/raw episodes available for a show.
func (c *AllAnimeProvider) GetAvailableEpisodes(ctx context.Context, showID string) (AvailableEpisodes, error) {
graphqlQuery := `query($showId: String!) {
show(_id: $showId) {
availableEpisodesDetail
lastEpisodeInfo
}
}`
result, err := c.graphqlRequest(ctx, graphqlQuery, map[string]any{"showId": showID})
if err != nil {
return AvailableEpisodes{}, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid response")
}
show, ok := data["show"].(map[string]any)
if !ok || show == nil {
return AvailableEpisodes{}, fmt.Errorf("show not found")
}
detail, ok := show["availableEpisodesDetail"].(map[string]any)
if !ok {
return AvailableEpisodes{}, fmt.Errorf("invalid detail")
}
var count AvailableEpisodes
if sub, ok := detail["sub"].([]any); ok {
for _, s := range sub {
if str, ok := s.(string); ok {
count.Sub = append(count.Sub, str)
}
}
}
if dub, ok := detail["dub"].([]any); ok {
for _, s := range dub {
if str, ok := s.(string); ok {
count.Dub = append(count.Dub, str)
}
}
}
if raw, ok := detail["raw"].([]any); ok {
for _, s := range raw {
if str, ok := s.(string); ok {
count.Raw = append(count.Raw, str)
}
}
}
return count, nil
}
func decodeSourceURL(encoded string) string {
if encoded == "" {
return ""
}
encoded = strings.TrimPrefix(encoded, "--")
substitutions := map[string]string{
"79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E",
"7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J",
"73": "K", "74": "L", "75": "M", "76": "N", "77": "O",
"68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T",
"6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y",
"62": "Z",
"59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e",
"5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j",
"53": "k", "54": "l", "55": "m", "56": "n", "57": "o",
"48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t",
"4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y",
"42": "z",
"08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4",
"0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9",
"15": "-", "16": ".", "67": "_", "46": "~", "02": ":",
"17": "/", "07": "?", "1b": "#", "63": "[", "65": "]",
"78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(",
"11": ")", "12": "*", "13": "+", "14": ",", "03": ";",
"05": "=", "1d": "%",
}
var result strings.Builder
for idx := 0; idx < len(encoded); {
if idx+2 <= len(encoded) {
pair := encoded[idx : idx+2]
if sub, ok := substitutions[pair]; ok {
result.WriteString(sub)
idx += 2
continue
}
}
result.WriteByte(encoded[idx])
idx++
}
decoded := result.String()
if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") {
decoded = strings.Replace(decoded, "/clock", "/clock.json", 1)
}
return decoded
}
func detectStreamType(sourceURL string) string {
lower := strings.ToLower(sourceURL)
if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") {
return "m3u8"
}
if strings.Contains(lower, ".mp4") {
return "mp4"
}
return "unknown"
}
func detectEmbedType(rawURL string) string {
lower := strings.ToLower(rawURL)
embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"}
for _, host := range embedHosts {
if strings.Contains(lower, host) {
return "embed"
}
}
return "unknown"
} }

View File

@@ -20,167 +20,182 @@ func isLikelyMP4(data []byte) bool {
return string(data[4:8]) == "ftyp" return string(data[4:8]) == "ftyp"
} }
func TestDecodeSourceURL(t *testing.T) { type stringTransformTestCase struct {
t.Parallel() name string
input string
want string
}
tests := []struct { type sourceReferencesTestCase struct {
name string name string
encoded string rawURLs []any
want string wantRefs []sourceReference
}{ }
{
name: "empty returns empty", func runStringTransformTests(t *testing.T, tests []stringTransformTestCase, fn func(string) string) {
encoded: "", t.Helper()
want: "",
},
{
name: "with double prefix stripped",
encoded: "--example.com/video.mp4",
want: "example.com/video.mp4",
},
{
name: "hex substitution",
encoded: "7aexample",
want: "Bexample",
},
{
name: "mixed substitution",
encoded: "79url7a01",
want: "AurlB9",
},
{
name: "clock replacement",
encoded: "/clock",
want: "/clock.json",
},
{
name: "no clock replacement if already json",
encoded: "/clock.json",
want: "/clock.json",
},
{
name: "complex url",
encoded: "--79stream7acom",
want: "AstreamBcom",
},
}
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
got := decodeSourceURL(tt.encoded) got := fn(tt.input)
if got != tt.want { if got != tt.want {
t.Errorf("decodeSourceURL(%q) = %q, want %q", tt.encoded, got, tt.want) t.Errorf("got %q for input %q, want %q", got, tt.input, tt.want)
} }
}) })
} }
} }
func runSourceReferenceTests(t *testing.T, tests []sourceReferencesTestCase) {
t.Helper()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := buildSourceReferences(tt.rawURLs)
if len(got) != len(tt.wantRefs) {
t.Errorf("got %d refs, want %d", len(got), len(tt.wantRefs))
return
}
for i, want := range tt.wantRefs {
if got[i].URL != want.URL {
t.Errorf("ref[%d].URL = %q, want %q", i, got[i].URL, want.URL)
}
if got[i].Name != want.Name {
t.Errorf("ref[%d].Name = %q, want %q", i, got[i].Name, want.Name)
}
}
})
}
}
func TestDecodeSourceURL(t *testing.T) {
t.Parallel()
tests := []stringTransformTestCase{
{
name: "empty returns empty",
input: "",
want: "",
},
{
name: "with double prefix stripped",
input: "--example.com/video.mp4",
want: "example.com/video.mp4",
},
{
name: "hex substitution",
input: "7aexample",
want: "Bexample",
},
{
name: "mixed substitution",
input: "79url7a01",
want: "AurlB9",
},
{
name: "clock replacement",
input: "/clock",
want: "/clock.json",
},
{
name: "no clock replacement if already json",
input: "/clock.json",
want: "/clock.json",
},
{
name: "complex url",
input: "--79stream7acom",
want: "AstreamBcom",
},
}
runStringTransformTests(t, tests, decodeSourceURL)
}
func TestDetectStreamType(t *testing.T) { func TestDetectStreamType(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []stringTransformTestCase{
name string
url string
wantType string
}{
{ {
name: "m3u8 extension", name: "m3u8 extension",
url: "https://example.com/video.m3u8", input: "https://example.com/video.m3u8",
wantType: "m3u8", want: "m3u8",
}, },
{ {
name: "master m3u8", name: "master m3u8",
url: "https://example.com/master.m3u8", input: "https://example.com/master.m3u8",
wantType: "m3u8", want: "m3u8",
}, },
{ {
name: "mp4 extension", name: "mp4 extension",
url: "https://example.com/video.mp4", input: "https://example.com/video.mp4",
wantType: "mp4", want: "mp4",
}, },
{ {
name: "unknown", name: "unknown",
url: "https://example.com/video.avi", input: "https://example.com/video.avi",
wantType: "unknown", want: "unknown",
}, },
{ {
name: "empty returns unknown", name: "empty returns unknown",
url: "", input: "",
wantType: "unknown", want: "unknown",
}, },
{ {
name: "case insensitive - M3U8", name: "case insensitive - M3U8",
url: "https://example.com/MASTER.M3U8", input: "https://example.com/MASTER.M3U8",
wantType: "m3u8", want: "m3u8",
}, },
} }
for _, tt := range tests { runStringTransformTests(t, tests, detectStreamType)
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := detectStreamType(tt.url)
if got != tt.wantType {
t.Errorf("detectStreamType(%q) = %q, want %q", tt.url, got, tt.wantType)
}
})
}
} }
func TestDetectEmbedType(t *testing.T) { func TestDetectEmbedType(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []stringTransformTestCase{
name string
url string
wantType string
}{
{ {
name: "streamwish", name: "streamwish",
url: "https://streamwish.com/e/abc123", input: "https://streamwish.com/e/abc123",
wantType: "embed", want: "embed",
}, },
{ {
name: "streamsb", name: "streamsb",
url: "https://streamsb.com/e/abc123", input: "https://streamsb.com/e/abc123",
wantType: "embed", want: "embed",
}, },
{ {
name: "mp4upload", name: "mp4upload",
url: "https://mp4upload.com/e/abc123", input: "https://mp4upload.com/e/abc123",
wantType: "embed", want: "embed",
}, },
{ {
name: "ok.ru", name: "ok.ru",
url: "https://ok.ru/video/123", input: "https://ok.ru/video/123",
wantType: "embed", want: "embed",
}, },
{ {
name: "gogoplay", name: "gogoplay",
url: "https://gogoplay.io/embed/123", input: "https://gogoplay.io/embed/123",
wantType: "embed", want: "embed",
}, },
{ {
name: "streamlare", name: "streamlare",
url: "https://streamlare.com/e/abc", input: "https://streamlare.com/e/abc",
wantType: "embed", want: "embed",
}, },
{ {
name: "unknown host", name: "unknown host",
url: "https://unknown.com/video", input: "https://unknown.com/video",
wantType: "unknown", want: "unknown",
}, },
} }
for _, tt := range tests { runStringTransformTests(t, tests, detectEmbedType)
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := detectEmbedType(tt.url)
if got != tt.wantType {
t.Errorf("detectEmbedType(%q) = %q, want %q", tt.url, got, tt.wantType)
}
})
}
} }
func TestBuildStreamSource(t *testing.T) { func TestBuildStreamSource(t *testing.T) {
@@ -204,14 +219,21 @@ func TestBuildStreamSource(t *testing.T) {
}) })
} }
func TestResolveDirectSourceSkipsEmbeds(t *testing.T) {
t.Parallel()
if _, ok := resolveDirectSource(sourceReference{
URL: "https://ok.ru/videoembed/123",
Name: "ok",
}); ok {
t.Fatal("expected embed URL to require extraction")
}
}
func TestBuildSourceReferences(t *testing.T) { func TestBuildSourceReferences(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []sourceReferencesTestCase{
name string
rawURLs []any
wantRefs []sourceReference
}{
{ {
name: "empty returns empty", name: "empty returns empty",
rawURLs: nil, rawURLs: nil,
@@ -263,26 +285,7 @@ func TestBuildSourceReferences(t *testing.T) {
}, },
} }
for _, tt := range tests { runSourceReferenceTests(t, tests)
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := buildSourceReferences(tt.rawURLs)
if len(got) != len(tt.wantRefs) {
t.Errorf("got %d refs, want %d", len(got), len(tt.wantRefs))
return
}
for i, want := range tt.wantRefs {
if got[i].URL != want.URL {
t.Errorf("ref[%d].URL = %q, want %q", i, got[i].URL, want.URL)
}
if got[i].Name != want.Name {
t.Errorf("ref[%d].Name = %q, want %q", i, got[i].Name, want.Name)
}
}
})
}
} }
func TestBuildSourceReferencesOrder(t *testing.T) { func TestBuildSourceReferencesOrder(t *testing.T) {
@@ -391,6 +394,27 @@ func TestIsLikelyMP4(t *testing.T) {
} }
} }
func TestParseOKRUSources(t *testing.T) {
t.Parallel()
body := `{"flashvars":{"metadata":"{\"hlsManifestUrl\":\"https://vd.example.test/video.m3u8?cmd=videoPlayerCdn\\u0026id=123\"}"}}`
got := parseOKRUSources(body, allAnimeReferer)
if len(got) != 1 {
t.Fatalf("len(got) = %d, want 1", len(got))
}
if got[0].URL != "https://vd.example.test/video.m3u8?cmd=videoPlayerCdn&id=123" {
t.Fatalf("URL = %q", got[0].URL)
}
if got[0].Type != "m3u8" {
t.Fatalf("Type = %q, want m3u8", got[0].Type)
}
if got[0].Provider != "ok" {
t.Fatalf("Provider = %q, want ok", got[0].Provider)
}
}
func TestDecryptTobeparsed(t *testing.T) { func TestDecryptTobeparsed(t *testing.T) {
t.Parallel() t.Parallel()

View File

@@ -0,0 +1,235 @@
package allanime
import (
"crypto/aes"
"crypto/cipher"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
)
var (
aesKeys = []string{"Xot36i3lK3:v1", "SimtVuagFbGR2K7P"}
)
func decryptTobeparsed(encoded string) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
return nil, fmt.Errorf("base64 decode failed: %w", err)
}
if len(raw) < 29 {
return nil, fmt.Errorf("encrypted payload too short")
}
version := raw[0]
iv := raw[1:13]
cipherText := raw[13 : len(raw)-16]
for _, keyStr := range aesKeys {
key := sha256.Sum256([]byte(keyStr))
block, err := aes.NewCipher(key[:])
if err != nil {
continue
}
if version == 1 {
plainText := tryDecryptCTR(block, iv, cipherText)
if json.Valid(plainText) {
return plainText, nil
}
}
gcm, err := cipher.NewGCM(block)
if err == nil {
tag := raw[len(raw)-16:]
combined := append(append([]byte{}, cipherText...), tag...)
plainText, openErr := gcm.Open(nil, iv, combined, nil)
if openErr == nil && json.Valid(plainText) {
return plainText, nil
}
}
}
return nil, fmt.Errorf("decryption failed")
}
func tryDecryptCTR(block cipher.Block, iv []byte, cipherText []byte) []byte {
ctrIV := append([]byte{}, iv...)
ctrIV = append(ctrIV, 0x00, 0x00, 0x00, 0x02)
ctr := cipher.NewCTR(block, ctrIV)
plainText := make([]byte, len(cipherText))
ctr.XORKeyStream(plainText, cipherText)
return plainText
}
func decodeSourceURL(encoded string) string {
if encoded == "" {
return ""
}
encoded = strings.TrimPrefix(encoded, "--")
substitutions := map[string]string{
"79": "A", "7a": "B", "7b": "C", "7c": "D", "7d": "E",
"7e": "F", "7f": "G", "70": "H", "71": "I", "72": "J",
"73": "K", "74": "L", "75": "M", "76": "N", "77": "O",
"68": "P", "69": "Q", "6a": "R", "6b": "S", "6c": "T",
"6d": "U", "6e": "V", "6f": "W", "60": "X", "61": "Y",
"62": "Z",
"59": "a", "5a": "b", "5b": "c", "5c": "d", "5d": "e",
"5e": "f", "5f": "g", "50": "h", "51": "i", "52": "j",
"53": "k", "54": "l", "55": "m", "56": "n", "57": "o",
"48": "p", "49": "q", "4a": "r", "4b": "s", "4c": "t",
"4d": "u", "4e": "v", "4f": "w", "40": "x", "41": "y",
"42": "z",
"08": "0", "09": "1", "0a": "2", "0b": "3", "0c": "4",
"0d": "5", "0e": "6", "0f": "7", "00": "8", "01": "9",
"15": "-", "16": ".", "67": "_", "46": "~", "02": ":",
"17": "/", "07": "?", "1b": "#", "63": "[", "65": "]",
"78": "@", "19": "!", "1c": "$", "1e": "&", "10": "(",
"11": ")", "12": "*", "13": "+", "14": ",", "03": ";",
"05": "=", "1d": "%",
}
var result strings.Builder
for idx := 0; idx < len(encoded); {
if idx+2 <= len(encoded) {
pair := encoded[idx : idx+2]
if sub, ok := substitutions[pair]; ok {
result.WriteString(sub)
idx += 2
continue
}
}
result.WriteByte(encoded[idx])
idx++
}
decoded := result.String()
if strings.Contains(decoded, "/clock") && !strings.Contains(decoded, "/clock.json") {
decoded = strings.Replace(decoded, "/clock", "/clock.json", 1)
}
return decoded
}
func responseFromTobeparsed(data map[string]any) (map[string]any, error) {
toBeParsed := firstNonEmptyString(
nestedString(data, "tobeparsed"),
nestedString(data, "episode", "tobeparsed"),
)
if toBeParsed == "" {
return nil, nil
}
decrypted, err := decryptTobeparsed(toBeParsed)
if err != nil {
return nil, fmt.Errorf("decrypt tobeparsed: %w", err)
}
parsed, err := parseGraphQLResponse(decrypted, "unmarshal decrypted")
if err != nil {
return nil, err
}
sourceURLs := firstNonEmptySlice(
nestedSlice(parsed, "sourceUrls"),
nestedSlice(parsed, "episode", "sourceUrls"),
)
if len(sourceURLs) == 0 {
return nil, nil
}
return map[string]any{
"episode": map[string]any{
"sourceUrls": sourceURLs,
},
}, nil
}
func hasEpisodeSourceURLs(data map[string]any) bool {
return len(nestedSlice(data, "episode", "sourceUrls")) > 0
}
func parseGraphQLResponse(respBody []byte, decodeErrPrefix string) (map[string]any, error) {
var parsed map[string]any
if err := json.Unmarshal(respBody, &parsed); err != nil {
return nil, fmt.Errorf("%s: %w", decodeErrPrefix, err)
}
if errs, ok := parsed["errors"].([]any); ok && len(errs) > 0 {
return nil, fmt.Errorf("graphql error: %v", errs[0])
}
return parsed, nil
}
func firstNonEmptyString(values ...string) string {
for _, value := range values {
if value != "" {
return value
}
}
return ""
}
func firstNonEmptySlice(values ...[]any) []any {
for _, value := range values {
if len(value) > 0 {
return value
}
}
return nil
}
func nestedString(data map[string]any, path ...string) string {
value, ok := nestedValue(data, path...)
if !ok {
return ""
}
str, ok := value.(string)
if !ok {
return ""
}
return str
}
func nestedSlice(data map[string]any, path ...string) []any {
value, ok := nestedValue(data, path...)
if !ok {
return nil
}
slice, ok := value.([]any)
if !ok {
return nil
}
return slice
}
func nestedValue(data map[string]any, path ...string) (any, bool) {
var current any = data
for _, key := range path {
currentMap, ok := current.(map[string]any)
if !ok {
return nil, false
}
current, ok = currentMap[key]
if !ok {
return nil, false
}
}
return current, true
}

View File

@@ -4,6 +4,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"html"
"io" "io"
netutil "mal/pkg/net" netutil "mal/pkg/net"
"net/http" "net/http"
@@ -19,10 +20,27 @@ type providerExtractor struct {
referer string referer string
} }
type providerLinkItem struct {
link string
resolutionStr string
}
type providerHLSItem struct {
url string
hardsubLang string
}
type providerResponseData struct {
referer string
links []providerLinkItem
hls []providerHLSItem
subtitles []Subtitle
}
func newProviderExtractor() *providerExtractor { func newProviderExtractor() *providerExtractor {
return &providerExtractor{ return &providerExtractor{
httpClient: &http.Client{Timeout: 30 * time.Second}, httpClient: &http.Client{Timeout: 30 * time.Second},
baseURL: allAnimeBaseURL, baseURL: allAnimeSiteURL,
referer: allAnimeReferer, referer: allAnimeReferer,
} }
} }
@@ -63,65 +81,45 @@ func (e *providerExtractor) ExtractVideoLinks(ctx context.Context, providerPath
return e.parseProviderResponse(ctx, string(body)), nil return e.parseProviderResponse(ctx, string(body)), nil
} }
func (e *providerExtractor) ExtractEmbedVideoLinks(ctx context.Context, rawURL string) ([]StreamSource, error) {
resp, err := doProxiedRequest(ctx, e.httpClient, rawURL, e.referer)
if err != nil {
return nil, fmt.Errorf("fetch embed response: %w", err)
}
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil {
return nil, fmt.Errorf("read embed response: %w", err)
}
return parseExternalEmbedResponse(rawURL, string(body), e.referer), nil
}
// parseProviderResponse extracts stream sources from provider JSON response. // parseProviderResponse extracts stream sources from provider JSON response.
func (e *providerExtractor) parseProviderResponse(ctx context.Context, response string) []StreamSource { func (e *providerExtractor) parseProviderResponse(ctx context.Context, response string) []StreamSource {
sources := make([]StreamSource, 0)
providerReferer := e.referer
var root any var root any
if err := json.Unmarshal([]byte(response), &root); err != nil { if err := json.Unmarshal([]byte(response), &root); err != nil {
return sources return []StreamSource{}
} }
type linkItem struct { data := collectProviderResponseData(root, e.referer)
link string sources := buildProviderLinkSources(data.links, data.referer)
resolutionStr string sources = append(sources, e.buildProviderHLSSources(ctx, data.hls, data.referer)...)
}
type hlsItem struct {
url string
hardsubLang string
}
linkItems := make([]linkItem, 0) attachSubtitles(sources, data.subtitles)
hlsItems := make([]hlsItem, 0)
subtitles := make([]Subtitle, 0) return sources
}
func collectProviderResponseData(root any, fallbackReferer string) providerResponseData {
data := providerResponseData{referer: fallbackReferer}
var walk func(v any) var walk func(v any)
walk = func(v any) { walk = func(v any) {
switch x := v.(type) { switch x := v.(type) {
case map[string]any: case map[string]any:
if ref, ok := x["Referer"].(string); ok && strings.TrimSpace(ref) != "" { collectProviderMapData(x, &data)
providerReferer = strings.TrimSpace(ref)
}
if link, ok := x["link"].(string); ok {
if res, ok := x["resolutionStr"].(string); ok {
linkItems = append(linkItems, linkItem{link: link, resolutionStr: res})
}
}
if u, ok := x["url"].(string); ok {
if lang, ok := x["hardsub_lang"].(string); ok {
hlsItems = append(hlsItems, hlsItem{url: u, hardsubLang: lang})
}
}
if subs, ok := x["subtitles"].([]any); ok {
for _, sub := range subs {
obj, ok := sub.(map[string]any)
if !ok {
continue
}
lang, _ := obj["lang"].(string)
src, _ := obj["src"].(string)
lang = strings.TrimSpace(lang)
src = strings.TrimSpace(src)
if lang == "" || src == "" {
continue
}
subtitles = append(subtitles, Subtitle{Lang: lang, URL: src})
}
}
for _, child := range x { for _, child := range x {
walk(child) walk(child)
} }
@@ -133,42 +131,98 @@ func (e *providerExtractor) parseProviderResponse(ctx context.Context, response
} }
walk(root) walk(root)
if data.referer == "" {
if providerReferer == "" { data.referer = fallbackReferer
providerReferer = e.referer
} }
for _, item := range linkItems { return data
}
func collectProviderMapData(node map[string]any, data *providerResponseData) {
if ref, ok := node["Referer"].(string); ok {
if trimmedRef := strings.TrimSpace(ref); trimmedRef != "" {
data.referer = trimmedRef
}
}
if link, ok := node["link"].(string); ok {
if res, ok := node["resolutionStr"].(string); ok {
data.links = append(data.links, providerLinkItem{link: link, resolutionStr: res})
}
}
if url, ok := node["url"].(string); ok {
if lang, ok := node["hardsub_lang"].(string); ok {
data.hls = append(data.hls, providerHLSItem{url: url, hardsubLang: lang})
}
}
if subs, ok := node["subtitles"].([]any); ok {
data.subtitles = append(data.subtitles, parseProviderSubtitles(subs)...)
}
}
func parseProviderSubtitles(items []any) []Subtitle {
subtitles := make([]Subtitle, 0, len(items))
for _, item := range items {
node, ok := item.(map[string]any)
if !ok {
continue
}
lang, _ := node["lang"].(string)
src, _ := node["src"].(string)
lang = strings.TrimSpace(lang)
src = strings.TrimSpace(src)
if lang == "" || src == "" {
continue
}
subtitles = append(subtitles, Subtitle{Lang: lang, URL: src})
}
return subtitles
}
func buildProviderLinkSources(items []providerLinkItem, referer string) []StreamSource {
sources := make([]StreamSource, 0, len(items))
for _, item := range items {
link := strings.TrimSpace(item.link) link := strings.TrimSpace(item.link)
if link == "" { if link == "" {
continue continue
} }
quality := strings.TrimSpace(item.resolutionStr)
sourceType := detectStreamType(link)
if sourceType == "unknown" {
sourceType = detectEmbedType(link)
}
sources = append(sources, StreamSource{ sources = append(sources, StreamSource{
URL: link, URL: link,
Quality: quality, Quality: strings.TrimSpace(item.resolutionStr),
Provider: "wixmp", Provider: "wixmp",
Type: sourceType, Type: detectProviderSourceType(link),
Referer: providerReferer, Referer: referer,
}) })
} }
for _, item := range hlsItems { return sources
if strings.TrimSpace(item.url) == "" { }
continue
} func detectProviderSourceType(link string) string {
if item.hardsubLang != "en-US" { sourceType := detectStreamType(link)
if sourceType != "unknown" {
return sourceType
}
return detectEmbedType(link)
}
func (e *providerExtractor) buildProviderHLSSources(ctx context.Context, items []providerHLSItem, referer string) []StreamSource {
sources := make([]StreamSource, 0, len(items))
for _, item := range items {
playlistURL, ok := providerPlaylistURL(item)
if !ok {
continue continue
} }
playlistURL := strings.TrimSpace(item.url)
if strings.Contains(playlistURL, "master.m3u8") { if strings.Contains(playlistURL, "master.m3u8") {
parsed, err := e.parseM3U8(ctx, playlistURL, providerReferer) parsed, err := e.parseM3U8(ctx, playlistURL, referer)
if err == nil { if err == nil {
sources = append(sources, parsed...) sources = append(sources, parsed...)
} }
@@ -180,17 +234,30 @@ func (e *providerExtractor) parseProviderResponse(ctx context.Context, response
Quality: "auto", Quality: "auto",
Provider: "hls", Provider: "hls",
Type: "m3u8", Type: "m3u8",
Referer: providerReferer, Referer: referer,
}) })
} }
if len(subtitles) > 0 && len(sources) > 0 { return sources
for idx := range sources { }
sources[idx].Subtitles = append([]Subtitle(nil), subtitles...)
} func providerPlaylistURL(item providerHLSItem) (string, bool) {
playlistURL := strings.TrimSpace(item.url)
if playlistURL == "" || item.hardsubLang != "en-US" {
return "", false
} }
return sources return playlistURL, true
}
func attachSubtitles(sources []StreamSource, subtitles []Subtitle) {
if len(subtitles) == 0 || len(sources) == 0 {
return
}
for idx := range sources {
sources[idx].Subtitles = append([]Subtitle(nil), subtitles...)
}
} }
// parseM3U8 fetches a master playlist and extracts individual stream URLs with bandwidth-derived quality. // parseM3U8 fetches a master playlist and extracts individual stream URLs with bandwidth-derived quality.
@@ -206,60 +273,159 @@ func (e *providerExtractor) parseM3U8(ctx context.Context, masterURL string, ref
return nil, err return nil, err
} }
lines := strings.Split(string(body), "\n") return parseM3U8Sources(string(body), masterURL, referer), nil
baseURL := masterURL }
if idx := strings.LastIndex(masterURL, "/"); idx >= 0 {
baseURL = masterURL[:idx+1]
}
func parseM3U8Sources(body string, masterURL string, referer string) []StreamSource {
lines := strings.Split(body, "\n")
baseURL := playlistBaseURL(masterURL)
bwPattern := regexp.MustCompile(`BANDWIDTH=(\d+)`)
currentBandwidth := 0 currentBandwidth := 0
sources := make([]StreamSource, 0) sources := make([]StreamSource, 0)
bwPattern := regexp.MustCompile(`BANDWIDTH=(\d+)`)
for _, line := range lines { for _, line := range lines {
trimmed := strings.TrimSpace(line) trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "#EXT-X-STREAM-INF") { if bandwidth, ok := parseStreamBandwidth(trimmed, bwPattern); ok {
match := bwPattern.FindStringSubmatch(trimmed) currentBandwidth = bandwidth
if len(match) >= 2 {
value, convErr := strconv.Atoi(match[1])
if convErr == nil {
currentBandwidth = value
}
}
continue continue
} }
if shouldSkipM3U8Line(trimmed) {
// skip empty lines and non-stream lines
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
continue continue
} }
streamURL := trimmed
if !strings.HasPrefix(streamURL, "http://") && !strings.HasPrefix(streamURL, "https://") {
streamURL = baseURL + streamURL
}
quality := "auto"
kbps := currentBandwidth / 1000
switch {
case kbps >= 8000:
quality = "1080p"
case kbps >= 5000:
quality = "720p"
case kbps >= 2500:
quality = "480p"
case kbps > 0:
quality = "360p"
}
sources = append(sources, StreamSource{ sources = append(sources, StreamSource{
URL: streamURL, URL: resolvePlaylistURL(trimmed, baseURL),
Quality: quality, Quality: qualityFromBandwidth(currentBandwidth),
Provider: "hls", Provider: "hls",
Type: "m3u8", Type: "m3u8",
Referer: referer, Referer: referer,
}) })
} }
return sources, nil return sources
}
func playlistBaseURL(masterURL string) string {
if idx := strings.LastIndex(masterURL, "/"); idx >= 0 {
return masterURL[:idx+1]
}
return masterURL
}
func parseStreamBandwidth(line string, bwPattern *regexp.Regexp) (int, bool) {
if !strings.HasPrefix(line, "#EXT-X-STREAM-INF") {
return 0, false
}
match := bwPattern.FindStringSubmatch(line)
if len(match) < 2 {
return 0, true
}
value, err := strconv.Atoi(match[1])
if err != nil {
return 0, true
}
return value, true
}
func shouldSkipM3U8Line(line string) bool {
return line == "" || strings.HasPrefix(line, "#")
}
func resolvePlaylistURL(streamURL string, baseURL string) string {
if strings.HasPrefix(streamURL, "http://") || strings.HasPrefix(streamURL, "https://") {
return streamURL
}
return baseURL + streamURL
}
func qualityFromBandwidth(bandwidth int) string {
kbps := bandwidth / 1000
switch {
case kbps >= 8000:
return "1080p"
case kbps >= 5000:
return "720p"
case kbps >= 2500:
return "480p"
case kbps > 0:
return "360p"
default:
return "auto"
}
}
func parseExternalEmbedResponse(rawURL string, body string, fallbackReferer string) []StreamSource {
switch {
case strings.Contains(strings.ToLower(rawURL), "ok.ru/"):
return parseOKRUSources(body, fallbackReferer)
case strings.Contains(strings.ToLower(rawURL), "mp4upload.com/"):
return parseMP4UploadSources(body, fallbackReferer)
default:
return nil
}
}
func parseOKRUSources(body string, referer string) []StreamSource {
unescapedBody := html.UnescapeString(body)
manifestPattern := regexp.MustCompile(`\\"hlsManifestUrl\\":\\"([^"]+)\\"|"hlsManifestUrl":"([^"]+)"`)
match := manifestPattern.FindStringSubmatch(unescapedBody)
if len(match) < 3 {
return nil
}
playlistURL := decodeEscapedMediaURL(firstNonEmptyString(match[1], match[2]))
if playlistURL == "" {
return nil
}
return []StreamSource{{
URL: playlistURL,
Quality: "auto",
Provider: "ok",
Type: "m3u8",
Referer: referer,
}}
}
func parseMP4UploadSources(body string, referer string) []StreamSource {
srcPattern := regexp.MustCompile(`(?m)src:\s*"([^"]+)"`)
match := srcPattern.FindStringSubmatch(body)
if len(match) < 2 {
return nil
}
mediaURL := decodeEscapedMediaURL(match[1])
if mediaURL == "" {
return nil
}
return []StreamSource{{
URL: mediaURL,
Provider: "mp4upload",
Type: detectProviderSourceType(mediaURL),
Referer: referer,
}}
}
func decodeEscapedMediaURL(raw string) string {
if unquoted, err := strconv.Unquote(`"` + raw + `"`); err == nil {
raw = unquoted
}
replacer := strings.NewReplacer(
`\\u002F`, `/`,
`\\u0026`, "&",
`\/`, `/`,
`\u002F`, `/`,
`\u0026`, "&",
`&amp;`, "&",
)
return strings.TrimSpace(replacer.Replace(raw))
} }

View File

@@ -0,0 +1,156 @@
package allanime
import (
"context"
"fmt"
"mal/pkg"
netutil "mal/pkg/net"
"strconv"
"strings"
)
const searchQuery = `query(
$search: SearchInput
$translationType: VaildTranslationTypeEnumType
$limit: Int = 40
$page: Int = 1
$countryOrigin: VaildCountryOriginEnumType = ALL
) {
shows(
search: $search
limit: $limit
page: $page
translationType: $translationType
countryOrigin: $countryOrigin
) {
edges {
_id
malId
name
}
}
}`
type searchResult struct {
ID string
MalID string
Name string
}
func (c *AllAnimeProvider) Search(ctx context.Context, query string, mode string) ([]searchResult, error) {
type searchData struct {
Shows struct {
Edges []struct {
ID string `json:"_id"`
MalID string `json:"malId"`
Name string `json:"name"`
} `json:"edges"`
} `json:"shows"`
}
type searchInput struct {
AllowAdult bool `json:"allowAdult"`
AllowUnknown bool `json:"allowUnknown"`
Query string `json:"query"`
}
type searchVariables struct {
Search searchInput `json:"search"`
TranslationType string `json:"translationType"`
}
vars := searchVariables{
Search: searchInput{
AllowAdult: false,
AllowUnknown: false,
Query: query,
},
TranslationType: mode,
}
data, err := graphql.Post[searchData](ctx, c.httpClient, allAnimeBaseURL+"/api", searchQuery, vars, graphql.PostOptions{
Headers: map[string]string{
"Referer": allAnimeReferer,
"User-Agent": defaultUserAgent,
},
BodyMax: netutil.MiB2,
})
if err != nil {
return nil, err
}
out := make([]searchResult, 0, len(data.Shows.Edges))
for _, edge := range data.Shows.Edges {
id := edge.ID
malID := edge.MalID
name := edge.Name
if unquoted, err := strconv.Unquote("\"" + name + "\""); err == nil {
name = unquoted
}
name = strings.TrimSpace(name)
if id == "" {
continue
}
out = append(out, searchResult{ID: id, MalID: malID, Name: name})
}
return out, nil
}
func (c *AllAnimeProvider) resolveShowIDWithFallback(ctx context.Context, animeID int, titleCandidates []string, mode string) string {
targetMalIDStr := strconv.Itoa(animeID)
firstAvailableShowID := ""
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil || len(searchResults) == 0 {
continue
}
if showID := exactMatchShowID(searchResults, targetMalIDStr); showID != "" {
return showID
}
if firstAvailableShowID == "" {
firstAvailableShowID = searchResults[0].ID
}
}
return firstAvailableShowID
}
func exactMatchShowID(searchResults []searchResult, targetMalID string) string {
for _, res := range searchResults {
if res.MalID == targetMalID {
return res.ID
}
}
return ""
}
func (c *AllAnimeProvider) ResolveEpisodeProviderID(ctx context.Context, animeID int, titleCandidates []string) (string, error) {
for _, mode := range []string{"sub", "dub"} {
showID, err := c.resolveShowIDStrict(ctx, animeID, titleCandidates, mode)
if err == nil {
return showID, nil
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d", animeID)
}
func (c *AllAnimeProvider) resolveShowIDStrict(ctx context.Context, animeID int, titleCandidates []string, mode string) (string, error) {
targetMalIDStr := strconv.Itoa(animeID)
for _, title := range titleCandidates {
searchResults, err := c.Search(ctx, title, mode)
if err != nil {
continue
}
for _, res := range searchResults {
if res.MalID == targetMalIDStr {
return res.ID, nil
}
}
}
return "", fmt.Errorf("allanime: no exact mal id match for %d in %s search", animeID, mode)
}

View File

@@ -0,0 +1,316 @@
package allanime
import (
"context"
"fmt"
"net/http"
"net/url"
"strings"
)
const episodeQueryHash = "d405d0edd690624b66baba3068e0edc3ac90f1597d898a1ec8db4e5c43c00fec"
type sourceReference struct {
URL string
Name string
}
func (c *AllAnimeProvider) GetEpisodeSources(ctx context.Context, showID string, episode string, mode string) ([]StreamSource, error) {
episodeQuery := `query($showId: String!, $translationType: VaildTranslationTypeEnumType!, $episodeString: String!) {
episode(showId: $showId, translationType: $translationType, episodeString: $episodeString) {
sourceUrls
}
}`
result, err := c.graphqlRequestWithHash(ctx, showID, episode, mode)
if err == nil {
sources := c.extractSourceURLsFromData(ctx, result)
if len(sources) > 0 {
return sources, nil
}
}
result, err = c.graphqlRequest(ctx, episodeQuery, map[string]any{
"showId": showID,
"translationType": mode,
"episodeString": episode,
})
if err != nil {
return nil, err
}
data, ok := result["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid source response")
}
rawSourceURLs, ok := data["episode"].(map[string]any)
if !ok {
return nil, fmt.Errorf("invalid episode response")
}
sourceURLs, ok := rawSourceURLs["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil, fmt.Errorf("no source urls")
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil, fmt.Errorf("no source references")
}
out := c.resolveSourceReferences(ctx, references)
if len(out) == 0 {
return nil, fmt.Errorf("no playable sources extracted")
}
return out, nil
}
func (c *AllAnimeProvider) extractSourceURLsFromData(ctx context.Context, data map[string]any) []StreamSource {
episodeData, ok := data["episode"].(map[string]any)
if !ok {
return nil
}
sourceURLs, ok := episodeData["sourceUrls"].([]any)
if !ok || len(sourceURLs) == 0 {
return nil
}
references := buildSourceReferences(sourceURLs)
if len(references) == 0 {
return nil
}
return c.resolveSourceReferences(ctx, references)
}
func (c *AllAnimeProvider) resolveSourceReferences(ctx context.Context, references []sourceReference) []StreamSource {
out := make([]StreamSource, 0, len(references))
for _, ref := range references {
if source, ok := resolveDirectSource(ref); ok {
out = append(out, source)
return out
}
extracted := c.resolveExtractedSources(ctx, ref)
if len(extracted) > 0 {
out = append(out, extracted...)
return out
}
}
return out
}
func resolveDirectSource(ref sourceReference) (StreamSource, bool) {
target := strings.TrimSpace(ref.URL)
if target == "" {
return StreamSource{}, false
}
if isHTTPURL(target) {
if detectEmbedType(target) == "embed" {
return StreamSource{}, false
}
return buildStreamSource(target, detectSourceType(target), ref.Name), true
}
decoded := decodeSourceURL(target)
if !isHTTPURL(decoded) {
return StreamSource{}, false
}
if detectEmbedType(decoded) == "embed" {
return StreamSource{}, false
}
return buildStreamSource(decoded, detectSourceType(decoded), ref.Name), true
}
func (c *AllAnimeProvider) resolveExtractedSources(ctx context.Context, ref sourceReference) []StreamSource {
rawURL := strings.TrimSpace(ref.URL)
decoded := decodeSourceURL(rawURL)
if decoded == "" {
return nil
}
if isHTTPURL(decoded) {
extracted, err := c.extractor.ExtractEmbedVideoLinks(ctx, decoded)
if err != nil {
return nil
}
return extracted
}
if !strings.HasPrefix(decoded, "/") {
decoded = "/" + decoded
}
extracted, err := c.extractor.ExtractVideoLinks(ctx, decoded)
if err != nil {
return nil
}
return extracted
}
func detectSourceType(sourceURL string) string {
sourceType := detectStreamType(sourceURL)
if sourceType != "unknown" {
return sourceType
}
return detectEmbedType(sourceURL)
}
func isHTTPURL(value string) bool {
return strings.HasPrefix(value, "http://") || strings.HasPrefix(value, "https://")
}
func buildStreamSource(url, sourceType, provider string) StreamSource {
return StreamSource{
URL: url,
Provider: provider,
Type: sourceType,
Referer: allAnimeReferer,
}
}
func buildSourceReferences(rawSourceURLs []any) []sourceReference {
priorityOrder := []string{"default", "yt-mp4", "s-mp4", "luf-mp4"}
prioritySet := map[string]struct{}{"default": {}, "yt-mp4": {}, "s-mp4": {}, "luf-mp4": {}}
prioritized := make(map[string]sourceReference)
fallback := make([]sourceReference, 0, len(rawSourceURLs))
seen := make(map[string]struct{})
for _, source := range rawSourceURLs {
item, ok := source.(map[string]any)
if !ok {
continue
}
sourceURL, _ := item["sourceUrl"].(string)
sourceName, _ := item["sourceName"].(string)
sourceURL = strings.TrimSpace(sourceURL)
sourceName = strings.TrimSpace(sourceName)
if sourceURL == "" {
continue
}
if _, exists := seen[sourceURL]; exists {
continue
}
seen[sourceURL] = struct{}{}
ref := sourceReference{URL: sourceURL, Name: sourceName}
normalized := strings.ToLower(sourceName)
if _, prioritizedProvider := prioritySet[normalized]; prioritizedProvider {
if _, exists := prioritized[normalized]; !exists {
prioritized[normalized] = ref
}
continue
}
fallback = append(fallback, ref)
}
ordered := make([]sourceReference, 0, len(prioritized)+len(fallback))
for _, provider := range priorityOrder {
if ref, ok := prioritized[provider]; ok {
ordered = append(ordered, ref)
}
}
ordered = append(ordered, fallback...)
return ordered
}
func (c *AllAnimeProvider) graphqlRequestWithHash(ctx context.Context, showID, episode, mode string) (map[string]any, error) {
req, err := newEpisodeHashRequest(ctx, showID, episode, mode)
if err != nil {
return nil, fmt.Errorf("create GET request: %w", err)
}
req.Header.Set("User-Agent", defaultUserAgent)
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Language", "en-US,en;q=0.5")
req.Header.Set("Accept-Encoding", "identity")
req.Header.Set("Referer", allAnimeReferer)
req.Header.Set("Origin", allAnimeOrigin)
req.Header.Set("Sec-Fetch-Dest", "empty")
req.Header.Set("Sec-Fetch-Mode", "cors")
req.Header.Set("Sec-Fetch-Site", "cross-site")
statusCode, respBody, err := executeAndReadResponse(c.utlsClient, req, "execute GET request", "read response")
if err != nil {
return nil, err
}
if statusCode != http.StatusOK {
return nil, fmt.Errorf("GET status %d: %s", statusCode, string(respBody))
}
parsed, err := parseGraphQLResponse(respBody, "decode response")
if err != nil {
return nil, err
}
data, ok := parsed["data"].(map[string]any)
if !ok {
return nil, fmt.Errorf("no data in response")
}
decrypted, err := responseFromTobeparsed(data)
if err != nil {
return nil, err
}
if decrypted != nil {
return decrypted, nil
}
if hasEpisodeSourceURLs(data) {
return parsed, nil
}
return nil, fmt.Errorf("no usable data in response")
}
func newEpisodeHashRequest(ctx context.Context, showID, episode, mode string) (*http.Request, error) {
varsJSON := fmt.Sprintf(`{"showId":"%s","translationType":"%s","episodeString":"%s"}`, showID, strings.ToLower(mode), episode)
extJSON := fmt.Sprintf(`{"persistedQuery":{"version":1,"sha256Hash":"%s"}}`, episodeQueryHash)
params := url.Values{}
params.Set("variables", varsJSON)
params.Set("extensions", extJSON)
return http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s/api?%s", allAnimeBaseURL, params.Encode()), nil)
}
func detectStreamType(sourceURL string) string {
lower := strings.ToLower(sourceURL)
if strings.Contains(lower, ".m3u8") || strings.Contains(lower, "master.m3u8") {
return "m3u8"
}
if strings.Contains(lower, ".mp4") {
return "mp4"
}
return "unknown"
}
func detectEmbedType(rawURL string) string {
lower := strings.ToLower(rawURL)
embedHosts := []string{"streamwish", "streamsb", "mp4upload", "ok.ru", "gogoplay", "streamlare"}
for _, host := range embedHosts {
if strings.Contains(lower, host) {
return "embed"
}
}
return "unknown"
}

View File

@@ -0,0 +1,391 @@
package anime
import (
"context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
)
type producerItem struct {
ID int `json:"id"`
Name string `json:"name"`
}
type browseQuery struct {
q string
animeType string
status string
orderBy string
sort string
sfw bool
studioID int
genres []int
page int
}
func producerQueryParams(c *gin.Context) (string, int, int, error) {
q := strings.TrimSpace(c.Query("q"))
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return "", 0, 0, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
limit, err := strconv.Atoi(c.DefaultQuery("limit", "50"))
if err != nil {
return "", 0, 0, fmt.Errorf("invalid limit")
}
if limit < 1 || limit > 12 {
limit = 12
}
return q, page, limit, nil
}
func producerItems(entries []jikan.ProducerListEntry) []producerItem {
items := make([]producerItem, 0, len(entries))
for _, producer := range entries {
name := jikan.ProducerListEntryName(producer)
if producer.MalID <= 0 || name == "" {
continue
}
items = append(items, producerItem{ID: producer.MalID, Name: name})
}
return items
}
func producerHTMLPayload(items []producerItem, hasNextPage bool, page int, q string, limit int) gin.H {
return gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": items,
"HasNextPage": hasNextPage,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
}
}
func requestWantsHTML(c *gin.Context) bool {
return strings.Contains(c.GetHeader("Accept"), "text/html")
}
func (h *AnimeHandler) HandleProducers(c *gin.Context) {
q, page, limit, err := producerQueryParams(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
res, err := h.svc.GetProducers(c.Request.Context(), q, page, limit)
if err != nil {
observability.WarnContext(c.Request.Context(),
"producers_fetch_failed",
"anime",
"",
map[string]any{
"q": q,
"page": page,
"limit": limit,
},
err,
)
if requestWantsHTML(c) {
c.HTML(http.StatusOK, "browse.gohtml", producerHTMLPayload([]producerItem{}, false, page, q, limit))
return
}
server.RespondError(
c,
http.StatusInternalServerError,
"producers_fetch_failed",
"anime",
"failed to load producers",
map[string]any{"q": q, "page": page, "limit": limit},
err,
)
return
}
items := producerItems(res.Items)
if requestWantsHTML(c) {
c.HTML(http.StatusOK, "browse.gohtml", producerHTMLPayload(items, res.HasNextPage, page, q, limit))
return
}
c.JSON(http.StatusOK, gin.H{
"items": items,
"hasNextPage": res.HasNextPage,
"nextPage": page + 1,
})
}
func parseBrowseQuery(c *gin.Context) (browseQuery, error) {
studioID := 0
if raw := strings.TrimSpace(c.Query("studio")); raw != "" {
id, err := strconv.Atoi(raw)
if err != nil || id < 0 {
return browseQuery{}, fmt.Errorf("invalid studio id")
}
studioID = id
}
genres := make([]int, 0, len(c.QueryArray("genres")))
for _, g := range c.QueryArray("genres") {
id, err := strconv.Atoi(g)
if err != nil {
return browseQuery{}, fmt.Errorf("invalid genre id")
}
if id > 0 {
genres = append(genres, id)
}
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return browseQuery{}, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
return browseQuery{
q: c.Query("q"),
animeType: c.Query("type"),
status: c.Query("status"),
orderBy: c.Query("order_by"),
sort: c.Query("sort"),
sfw: c.Query("sfw") != "false",
studioID: studioID,
genres: genres,
page: page,
}, nil
}
func browseStudioName(ctx context.Context, svc Service, studioID int) string {
if studioID <= 0 {
return ""
}
name, err := svc.GetProducerNameByID(ctx, studioID)
if err != nil {
return ""
}
return name
}
func browseTemplateData(
q browseQuery,
studioName string,
genresList []domain.Genre,
animes []domain.Anime,
user any,
watchlistMap map[int64]bool,
hasNextPage bool,
) gin.H {
return gin.H{
"CurrentPath": "/browse",
"Query": q.q,
"Type": q.animeType,
"Status": q.status,
"OrderBy": q.orderBy,
"Sort": q.sort,
"Genres": q.genres,
"Studio": q.studioID,
"StudioName": studioName,
"SFW": q.sfw,
"GenresList": genresList,
"Animes": animes,
"HasNextPage": hasNextPage,
"NextPage": q.page + 1,
"User": user,
"WatchlistMap": watchlistMap,
}
}
func (h *AnimeHandler) searchBrowse(ctx context.Context, query browseQuery) (jikan.SearchResult, error) {
return h.svc.SearchAdvanced(
ctx,
query.q,
query.animeType,
query.status,
query.orderBy,
query.sort,
query.genres,
query.studioID,
query.sfw,
query.page,
24,
)
}
func browseScrollData(
query browseQuery,
studioName string,
animes []domain.Anime,
watchlistMap map[int64]bool,
hasNextPage bool,
) gin.H {
return gin.H{
"_fragment": "anime_card_scroll",
"Animes": animes,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"Query": query.q,
"Type": query.animeType,
"Status": query.status,
"OrderBy": query.orderBy,
"Sort": query.sort,
"Genres": query.genres,
"Studio": query.studioID,
"StudioName": studioName,
"SFW": query.sfw,
"WatchlistMap": watchlistMap,
}
}
func (h *AnimeHandler) respondBrowseSearchError(c *gin.Context, query browseQuery, err error) {
server.RespondError(
c,
http.StatusInternalServerError,
"browse_search_failed",
"anime",
"failed to load browse results",
map[string]any{
"q": query.q,
"type": query.animeType,
"status": query.status,
"order_by": query.orderBy,
"sort": query.sort,
"studio": query.studioID,
"sfw": query.sfw,
"page": query.page,
},
err,
)
}
func (h *AnimeHandler) HandleBrowse(c *gin.Context) {
query, err := parseBrowseQuery(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
res, err := h.searchBrowse(c.Request.Context(), query)
if err != nil {
h.respondBrowseSearchError(c, query, err)
return
}
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
studioName := browseStudioName(c.Request.Context(), h.svc, query.studioID)
if c.GetHeader("HX-Request") == "true" && query.page > 1 {
c.HTML(http.StatusOK, "browse.gohtml", browseScrollData(query, studioName, animes, watchlistMap, res.HasNextPage))
return
}
genresList, _ := h.svc.GetGenres(c.Request.Context())
browseData := browseTemplateData(query, studioName, genresList, animes, user, watchlistMap, res.HasNextPage)
if c.GetHeader("HX-Request") == "true" {
browseData["_fragment"] = "browse_content"
c.HTML(http.StatusOK, "browse.gohtml", browseData)
return
}
c.HTML(http.StatusOK, "browse.gohtml", browseData)
}
type quickSearchResult struct {
ID int `json:"id"`
Title string `json:"title"`
Type string `json:"type"`
Year int `json:"year"`
Image string `json:"image"`
InWatchlist bool `json:"in_watchlist"`
}
func (h *AnimeHandler) HandleQuickSearch(c *gin.Context) {
query := c.Query("q")
if query == "" {
c.JSON(http.StatusOK, []any{})
return
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, 1, 5)
if err != nil {
c.JSON(http.StatusOK, []any{})
return
}
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
output := make([]quickSearchResult, len(animes))
for i, anime := range animes {
output[i] = quickSearchResult{
ID: anime.MalID,
Title: anime.DisplayTitle(),
Type: anime.Type,
Year: anime.Year,
Image: anime.ImageURL(),
InWatchlist: watchlistMap[int64(anime.MalID)],
}
}
c.JSON(http.StatusOK, output)
}
func (h *AnimeHandler) HandleRandomAnime(c *gin.Context) {
ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
defer cancel()
anime, err := h.svc.GetRandomAnime(ctx)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"random_anime_fetch_failed",
"anime",
"failed to fetch random anime",
nil,
err,
)
return
}
if anime.MalID == 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadGateway, "random anime unavailable")
return
}
inWatchlist := false
userID := server.CurrentUserID(c)
if userID != "" {
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, []int64{int64(anime.MalID)})
inWatchlist = watchlistMap[int64(anime.MalID)]
}
c.JSON(http.StatusOK, gin.H{
"data": anime,
"in_watchlist": inWatchlist,
})
}

View File

@@ -0,0 +1,123 @@
package anime
import (
"mal/internal/observability"
"mal/internal/server"
"net/http"
"github.com/gin-gonic/gin"
)
func (h *AnimeHandler) HandleSearch(c *gin.Context) {
c.HTML(http.StatusOK, "search.gohtml", gin.H{
"User": server.CurrentUser(c),
"CurrentPath": "/search",
})
}
func (h *AnimeHandler) HandleCatalog(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "index.gohtml", gin.H{
"CurrentPath": "/",
"User": user,
"WatchlistMap": map[int64]bool{},
})
}
func (h *AnimeHandler) HandleCatalogAiring(c *gin.Context) {
h.renderCatalogSection(c, "Airing")
}
func (h *AnimeHandler) HandleCatalogPopular(c *gin.Context) {
h.renderCatalogSection(c, "Popular")
}
func (h *AnimeHandler) HandleCatalogContinue(c *gin.Context) {
h.renderCatalogSection(c, "Continue")
}
func (h *AnimeHandler) HandleCatalogTopPickForYou(c *gin.Context) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPickForYou(c.Request.Context(), userID)
if err != nil {
observability.WarnContext(c.Request.Context(),
"top_pick_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = "TopPickForYou"
data.Fragment = "top_pick_for_you_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) renderCatalogSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetCatalogSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "catalog_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "catalog_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) HandleTopPicks(c *gin.Context) {
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPicksForYou(c.Request.Context(), userID)
if err != nil {
observability.WarnContext(c.Request.Context(),
"top_picks_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
c.HTML(http.StatusOK, "top_picks.gohtml", gin.H{
"CurrentPath": "/top-picks",
"User": user,
"Animes": data.Animes,
"WatchlistMap": watchlistMap,
})
}
func (h *AnimeHandler) abortSectionFetch(c *gin.Context, event string, userID string, section string, err error) {
observability.WarnContext(c.Request.Context(),
event,
"anime",
"",
map[string]any{
"section": section,
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
}

View File

@@ -1,19 +1,19 @@
package anime package anime
import ( import (
"context"
"fmt" "fmt"
"mal/internal/db" "mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/server" "mal/internal/server"
"net/http" "net/http"
"net/url" "strconv"
"strings" "strings"
"time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
) )
const commandPaletteAnimeLimit = 24
type commandPaletteItem struct { type commandPaletteItem struct {
ID string `json:"id"` ID string `json:"id"`
Type string `json:"type"` Type string `json:"type"`
@@ -24,6 +24,12 @@ type commandPaletteItem struct {
Icon string `json:"icon,omitempty"` Icon string `json:"icon,omitempty"`
} }
type commandPaletteResponse struct {
Items []commandPaletteItem `json:"items"`
HasNextPage bool `json:"hasNextPage"`
NextPage int `json:"nextPage,omitempty"`
}
func (h *AnimeHandler) HandleCommandPalette(c *gin.Context) { func (h *AnimeHandler) HandleCommandPalette(c *gin.Context) {
user := server.CurrentUser(c) user := server.CurrentUser(c)
if user == nil { if user == nil {
@@ -32,41 +38,49 @@ func (h *AnimeHandler) HandleCommandPalette(c *gin.Context) {
} }
query := strings.TrimSpace(c.Query("q")) query := strings.TrimSpace(c.Query("q"))
items := make([]commandPaletteItem, 0, 12) page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil || page < 1 {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid page"})
return
}
items := make([]commandPaletteItem, 0, commandPaletteAnimeLimit)
if query != "" { if query != "" {
items = append(items, commandPaletteItem{ hasNextPage := false
ID: "search:" + strings.ToLower(query),
Type: "search",
Label: fmt.Sprintf("Search anime for %q", query),
Subtitle: "Browse",
Href: "/browse?q=" + url.QueryEscape(query),
Icon: "search",
})
if len(query) >= 2 { if len(query) >= 2 {
items = append(items, h.commandPaletteAnimeResults(c, query)...) var animeItems []commandPaletteItem
animeItems, hasNextPage = h.commandPaletteAnimeResults(c, query, page)
items = append(items, animeItems...)
} }
items = append(items, h.commandPaletteNavigationItems(query)...) if page == 1 {
items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...) items = append(items, h.commandPaletteNavigationItems(query)...)
items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...) items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...)
c.JSON(http.StatusOK, items) items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...)
}
c.JSON(http.StatusOK, commandPaletteResponse{
Items: items,
HasNextPage: hasNextPage,
NextPage: page + 1,
})
return return
} }
items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...) items = append(items, h.commandPaletteContinueItems(c, user.ID, query)...)
items = append(items, h.commandPaletteNavigationItems(query)...) items = append(items, h.commandPaletteNavigationItems(query)...)
items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...) items = append(items, h.commandPalettePersonalItems(c, user.ID, query)...)
c.JSON(http.StatusOK, items) c.JSON(http.StatusOK, commandPaletteResponse{Items: items})
} }
func (h *AnimeHandler) commandPaletteNavigationItems(query string) []commandPaletteItem { func (h *AnimeHandler) commandPaletteNavigationItems(query string) []commandPaletteItem {
all := []commandPaletteItem{ all := []commandPaletteItem{
{ID: "nav:discover", Type: "navigation", Label: "Go to Discover", Subtitle: "Navigation", Href: "/discover", Icon: "compass"}, {ID: "nav:home", Type: "navigation", Label: "Go to Home", Subtitle: "Navigation", Href: "/", Icon: "home"},
{ID: "nav:watchlist", Type: "navigation", Label: "Go to Watchlist", Subtitle: "Navigation", Href: "/watchlist", Icon: "bookmark"}, {ID: "nav:watchlist", Type: "navigation", Label: "Go to Watchlist", Subtitle: "Navigation", Href: "/watchlist", Icon: "bookmark"},
{ID: "nav:popular", Type: "navigation", Label: "Browse popular", Subtitle: "Browse", Href: "/browse?order_by=popularity&sort=desc", Icon: "trending"}, {ID: "nav:top-picks", Type: "navigation", Label: "Open Top Picks", Subtitle: "Navigation", Href: "/top-picks", Icon: "sparkles"},
{ID: "nav:airing", Type: "navigation", Label: "Currently airing", Subtitle: "Browse", Href: "/browse?status=airing&order_by=popularity&sort=desc", Icon: "play"}, {ID: "nav:popular", Type: "navigation", Label: "Browse popular", Subtitle: "Browse", Href: "/browse?order_by=popularity&sort=asc", Icon: "trending"},
{ID: "nav:airing", Type: "navigation", Label: "Currently airing", Subtitle: "Browse", Href: "/browse?status=airing&order_by=popularity&sort=asc", Icon: "play"},
} }
if query == "" { if query == "" {
return all return all
@@ -81,13 +95,10 @@ func (h *AnimeHandler) commandPaletteNavigationItems(query string) []commandPale
return filtered return filtered
} }
func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string) []commandPaletteItem { func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string, page int) ([]commandPaletteItem, bool) {
searchCtx, cancel := context.WithTimeout(c.Request.Context(), 800*time.Millisecond) res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, page, commandPaletteAnimeLimit)
defer cancel()
res, err := h.svc.SearchAdvanced(searchCtx, query, "", "", "", "", nil, 0, true, 1, 5)
if err != nil { if err != nil {
return nil return nil, false
} }
animes := wrapAnimes(res.Animes) animes := wrapAnimes(res.Animes)
@@ -102,7 +113,7 @@ func (h *AnimeHandler) commandPaletteAnimeResults(c *gin.Context, query string)
Image: anime.ImageURL(), Image: anime.ImageURL(),
}) })
} }
return items return items, res.HasNextPage
} }
func (h *AnimeHandler) commandPalettePersonalItems(c *gin.Context, userID string, query string) []commandPaletteItem { func (h *AnimeHandler) commandPalettePersonalItems(c *gin.Context, userID string, query string) []commandPaletteItem {

View File

@@ -0,0 +1,220 @@
package anime
import (
"context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"time"
"github.com/gin-gonic/gin"
)
const (
animeSectionTimeout = 12 * time.Second
watchOrderTimeout = 15 * time.Second
audioLookupTimeout = 8 * time.Second
)
func animeAudioAvailabilityLabel(episodes []domain.CanonicalEpisode) string {
hasKnownSub := false
for _, episode := range episodes {
if episode.HasDub {
return "Dub available"
}
if episode.HasSub || episode.SubOnly {
hasKnownSub = true
}
}
if hasKnownSub {
return "Subtitled only"
}
return ""
}
func (h *AnimeHandler) animeAudioAvailability(ctx context.Context, anime domain.Anime) string {
if h.episodeSvc == nil {
return ""
}
audioCtx, cancel := context.WithTimeout(ctx, audioLookupTimeout)
defer cancel()
episodeList, err := h.episodeSvc.GetCanonicalEpisodes(audioCtx, anime, true)
if err != nil {
observability.Warn(
"anime_audio_availability_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return ""
}
if episodeList.Source != "AllAnime" {
return ""
}
return animeAudioAvailabilityLabel(episodeList.Episodes)
}
func (h *AnimeHandler) HandleAnimeDetails(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
section := c.Query("section")
if section != "" && c.GetHeader("HX-Request") == "true" {
h.handleAnimeDetailsSection(c, id, section)
return
}
anime, err := h.svc.GetAnimeByID(c.Request.Context(), id)
if err != nil {
c.Status(http.StatusNotFound)
return
}
h.svc.WarmDetailSections(id)
user := server.CurrentUser(c)
status := ""
var watchlistIDs []int64
ep := 0
var cwSeconds float64
if user != nil {
entry, err := h.watchlistSvc.GetWatchListEntry(c.Request.Context(), user.ID, int64(id))
if err == nil {
status = entry.Status
watchlistIDs = []int64{entry.AnimeID}
}
cwEntry, err := h.watchlistSvc.GetContinueWatchingEntry(c.Request.Context(), user.ID, int64(id))
if err == nil && cwEntry.CurrentEpisode.Valid {
ep = int(cwEntry.CurrentEpisode.Int64)
cwSeconds = cwEntry.CurrentTimeSeconds
}
}
audioAvailability := h.animeAudioAvailability(c.Request.Context(), anime)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"Anime": anime,
"AudioAvailability": audioAvailability,
"CurrentPath": fmt.Sprintf("/anime/%d", id),
"User": user,
"Status": status,
"WatchlistIDs": watchlistIDs,
"ContinueWatchingEp": ep,
"ContinueWatchingTime": cwSeconds,
})
}
func (h *AnimeHandler) handleAnimeDetailsSection(c *gin.Context, id int, section string) {
sectionCtx, cancel := context.WithTimeout(c.Request.Context(), animeSectionTimeout)
defer cancel()
data, tplName, err := h.loadAnimeDetailsSection(sectionCtx, id, section)
if err != nil {
observability.Warn(
"anime_section_fetch_failed",
"anime",
"",
map[string]any{
"section": section,
"anime_id": id,
},
err,
)
if section == "recommendations" {
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "anime_recommendations_loading",
"AnimeID": id,
})
return
}
c.Status(http.StatusNoContent)
return
}
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": tplName,
"Items": data,
})
}
func (h *AnimeHandler) loadAnimeDetailsSection(ctx context.Context, id int, section string) (any, string, error) {
switch section {
case "characters":
data, err := h.svc.GetCharacters(ctx, id)
return data, "anime_characters", err
case "recommendations":
data, err := h.svc.GetRecommendations(ctx, id)
return data, "anime_recommendations", err
case "statistics":
data, err := h.svc.GetStatistics(ctx, id)
return data, "anime_statistics", err
case "themes":
data, err := h.svc.GetThemes(ctx, id)
return data, "anime_themes", err
default:
return nil, "", nil
}
}
func (h *AnimeHandler) HandleHTMLWatchOrder(c *gin.Context) {
id, err := strconv.Atoi(c.Query("animeId"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
userID := server.CurrentUserID(c)
mode := jikan.NormalizeWatchOrderMode(c.Query("mode"))
relationsCtx, cancel := context.WithTimeout(c.Request.Context(), watchOrderTimeout)
defer cancel()
relations, err := h.svc.GetRelations(relationsCtx, id, mode)
if err != nil {
observability.Warn(
"relations_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": id,
},
err,
)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order_loading",
"AnimeID": id,
"Mode": string(mode),
})
return
}
relationAnimeIDs := make([]int64, 0, len(relations))
for _, relation := range relations {
if relation.Anime.MalID > 0 {
relationAnimeIDs = append(relationAnimeIDs, int64(relation.Anime.MalID))
}
}
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, relationAnimeIDs)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order",
"Relations": relations,
"AnimeID": id,
"Mode": string(mode),
"WatchlistMap": watchlistMap,
})
}

View File

@@ -2,38 +2,22 @@ package anime
import ( import (
"context" "context"
"fmt"
"mal/integrations/jikan"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability"
"mal/internal/server"
"net/http"
"strconv"
"strings"
"sync" "sync"
"time"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
) )
const (
animeSectionTimeout = 12 * time.Second
watchOrderTimeout = 15 * time.Second
audioLookupTimeout = 8 * time.Second
)
type AnimeHandler struct { type AnimeHandler struct {
svc Service svc Service
watchlistSvc domain.WatchlistService watchlistSvc domain.WatchlistService
episodeSvc domain.EpisodeService episodeSvc domain.EpisodeService
scheduleCache map[string]cachedWeekSchedule
scheduleCacheMu sync.Mutex sync.Mutex
scheduleCache map[string]cachedWeekSchedule
} }
type Service interface { type Service interface {
domain.AnimeCatalogService domain.AnimeCatalogService
domain.AnimeDiscoverService
domain.AnimeSearchService domain.AnimeSearchService
domain.AnimeDetailsService domain.AnimeDetailsService
WarmDetailSections(id int) WarmDetailSections(id int)
@@ -44,7 +28,7 @@ func NewAnimeHandler(svc Service, watchlistSvc domain.WatchlistService, episodeS
svc: svc, svc: svc,
watchlistSvc: watchlistSvc, watchlistSvc: watchlistSvc,
episodeSvc: episodeSvc, episodeSvc: episodeSvc,
scheduleCache: map[string]cachedWeekSchedule{}, scheduleCache: make(map[string]cachedWeekSchedule),
} }
} }
@@ -70,63 +54,14 @@ func (h *AnimeHandler) watchlistMapForIDs(ctx context.Context, userID string, an
return watchlistMap return watchlistMap
} }
func animeAudioAvailabilityLabel(episodes []domain.CanonicalEpisode) string {
hasKnownSub := false
for _, episode := range episodes {
if episode.HasDub {
return "Dub available"
}
if episode.HasSub || episode.SubOnly {
hasKnownSub = true
}
}
if hasKnownSub {
return "Subtitled only"
}
return ""
}
func (h *AnimeHandler) animeAudioAvailability(ctx context.Context, anime domain.Anime) string {
if h.episodeSvc == nil {
return ""
}
audioCtx, cancel := context.WithTimeout(ctx, audioLookupTimeout)
defer cancel()
episodeList, err := h.episodeSvc.GetCanonicalEpisodes(audioCtx, anime, true)
if err != nil {
observability.Warn(
"anime_audio_availability_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return ""
}
if episodeList.Source != "AllAnime" {
return ""
}
return animeAudioAvailabilityLabel(episodeList.Episodes)
}
func (h *AnimeHandler) Register(r *gin.Engine) { func (h *AnimeHandler) Register(r *gin.Engine) {
r.GET("/", h.HandleCatalog) r.GET("/", h.HandleCatalog)
r.GET("/api/catalog/airing", h.HandleCatalogAiring) r.GET("/api/catalog/airing", h.HandleCatalogAiring)
r.GET("/api/catalog/popular", h.HandleCatalogPopular) r.GET("/api/catalog/popular", h.HandleCatalogPopular)
r.GET("/api/catalog/continue", h.HandleCatalogContinue) r.GET("/api/catalog/continue", h.HandleCatalogContinue)
r.GET("/api/catalog/top-pick", h.HandleCatalogTopPickForYou) r.GET("/api/catalog/top-pick", h.HandleCatalogTopPickForYou)
r.GET("/discover", h.HandleDiscover) r.GET("/search", h.HandleSearch)
r.GET("/discover/top-picks", h.HandleDiscoverTopPicksForYou) r.GET("/top-picks", h.HandleTopPicks)
r.GET("/api/discover/trending", h.HandleDiscoverTrending)
r.GET("/api/discover/upcoming", h.HandleDiscoverUpcoming)
r.GET("/api/discover/top", h.HandleDiscoverTop)
r.GET("/schedule", h.HandleSchedule)
r.GET("/api/schedule", h.HandleScheduleSection)
r.GET("/browse", h.HandleBrowse) r.GET("/browse", h.HandleBrowse)
r.GET("/anime/:id", h.HandleAnimeDetails) r.GET("/anime/:id", h.HandleAnimeDetails)
r.GET("/anime/:id/reviews", h.HandleAnimeReviews) r.GET("/anime/:id/reviews", h.HandleAnimeReviews)
@@ -136,700 +71,3 @@ func (h *AnimeHandler) Register(r *gin.Engine) {
r.GET("/api/jikan/random/anime", h.HandleRandomAnime) r.GET("/api/jikan/random/anime", h.HandleRandomAnime)
r.GET("/api/jikan/producers", h.HandleProducers) r.GET("/api/jikan/producers", h.HandleProducers)
} }
func (h *AnimeHandler) HandleProducers(c *gin.Context) {
q := strings.TrimSpace(c.Query("q"))
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
limit, err := strconv.Atoi(c.DefaultQuery("limit", "50"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid limit")
return
}
if limit < 1 {
limit = 12
}
if limit > 12 {
limit = 12
}
res, err := h.svc.GetProducers(c.Request.Context(), q, page, limit)
if err != nil {
observability.Warn(
"producers_fetch_failed",
"anime",
"",
map[string]any{
"q": q,
"page": page,
"limit": limit,
},
err,
)
if strings.Contains(c.GetHeader("Accept"), "text/html") {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": []any{},
"HasNextPage": false,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
})
return
}
server.RespondError(
c,
http.StatusInternalServerError,
"producers_fetch_failed",
"anime",
"failed to load producers",
map[string]any{"q": q, "page": page, "limit": limit},
err,
)
return
}
type item struct {
ID int `json:"id"`
Name string `json:"name"`
}
items := make([]item, 0, len(res.Items))
for _, p := range res.Items {
name := jikan.ProducerListEntryName(p)
if p.MalID <= 0 || name == "" {
continue
}
items = append(items, item{ID: p.MalID, Name: name})
}
if strings.Contains(c.GetHeader("Accept"), "text/html") {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "studio_dropdown_items",
"StudioItems": items,
"HasNextPage": res.HasNextPage,
"Page": page,
"NextPage": page + 1,
"Query": q,
"Limit": limit,
})
return
}
c.JSON(http.StatusOK, gin.H{
"items": items,
"hasNextPage": res.HasNextPage,
"nextPage": page + 1,
})
}
func (h *AnimeHandler) HandleCatalog(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "index.gohtml", gin.H{
"CurrentPath": "/",
"User": user,
"WatchlistMap": map[int64]bool{},
})
}
func (h *AnimeHandler) HandleCatalogAiring(c *gin.Context) {
h.renderCatalogSection(c, "Airing")
}
func (h *AnimeHandler) HandleCatalogPopular(c *gin.Context) {
h.renderCatalogSection(c, "Popular")
}
func (h *AnimeHandler) HandleCatalogContinue(c *gin.Context) {
h.renderCatalogSection(c, "Continue")
}
func (h *AnimeHandler) HandleCatalogTopPickForYou(c *gin.Context) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPickForYou(c.Request.Context(), userID)
if err != nil {
observability.Warn(
"top_pick_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = "TopPickForYou"
data.Fragment = "top_pick_for_you_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) renderCatalogSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetCatalogSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "catalog_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "catalog_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "index.gohtml", data)
}
func (h *AnimeHandler) HandleDiscover(c *gin.Context) {
user := server.CurrentUser(c)
c.HTML(http.StatusOK, "discover.gohtml", gin.H{
"CurrentPath": "/discover",
"User": user,
})
}
func (h *AnimeHandler) HandleDiscoverTopPicksForYou(c *gin.Context) {
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
data, err := h.svc.GetTopPicksForYou(c.Request.Context(), userID)
if err != nil {
observability.Warn(
"top_picks_for_you_fetch_failed",
"anime",
"",
map[string]any{
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
c.HTML(http.StatusOK, "discover.gohtml", gin.H{
"_fragment": "",
"CurrentPath": "/discover",
"User": user,
"Animes": data.Animes,
"WatchlistMap": watchlistMap,
"IsTopPicks": true,
})
}
func (h *AnimeHandler) HandleDiscoverTrending(c *gin.Context) {
h.renderDiscoverSection(c, "Trending")
}
func (h *AnimeHandler) HandleDiscoverUpcoming(c *gin.Context) {
h.renderDiscoverSection(c, "Upcoming")
}
func (h *AnimeHandler) HandleDiscoverTop(c *gin.Context) {
h.renderDiscoverSection(c, "Top")
}
func (h *AnimeHandler) renderDiscoverSection(c *gin.Context, section string) {
userID := server.CurrentUserID(c)
data, err := h.svc.GetDiscoverSection(c.Request.Context(), userID, section)
if err != nil {
h.abortSectionFetch(c, "discover_section_fetch_failed", userID, section, err)
return
}
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, data.Animes)
data.Section = section
data.Fragment = "discover_section"
data.WatchlistMap = watchlistMap
c.HTML(http.StatusOK, "discover.gohtml", data)
}
func (h *AnimeHandler) abortSectionFetch(c *gin.Context, event string, userID string, section string, err error) {
observability.Warn(
event,
"anime",
"",
map[string]any{
"section": section,
"user_id": userID,
},
err,
)
c.AbortWithStatus(http.StatusInternalServerError)
}
func (h *AnimeHandler) HandleSchedule(c *gin.Context) {
user := server.CurrentUser(c)
year, week := parseYearWeek(c)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"CurrentPath": "/schedule",
"User": user,
"ScheduleYear": year,
"ScheduleWeek": week,
})
}
func (h *AnimeHandler) HandleScheduleSection(c *gin.Context) {
year, week := parseYearWeek(c)
timezone := scheduleTimezone(c)
schedule, err := h.getCachedAnimeScheduleWeek(c.Request.Context(), year, week, timezone)
if err != nil {
prevYear, prevWeek := adjacentISOWeek(year, week, -1)
nextYear, nextWeek := adjacentISOWeek(year, week, 1)
observability.Warn(
"animeschedule_fetch_failed",
"anime",
"",
map[string]any{
"year": year,
"week": week,
"timezone": timezone,
},
err,
)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"_fragment": "schedule_section_scraped",
"ScheduleDays": []any{},
"ScheduleYear": year,
"ScheduleWeek": week,
"PrevYear": prevYear,
"PrevWeek": prevWeek,
"NextYear": nextYear,
"NextWeek": nextWeek,
"ScheduleError": true,
})
return
}
days := buildScheduleDays(schedule, schedule.Year, schedule.Week)
prevYear, prevWeek := adjacentISOWeek(schedule.Year, schedule.Week, -1)
nextYear, nextWeek := adjacentISOWeek(schedule.Year, schedule.Week, 1)
c.HTML(http.StatusOK, "schedule.gohtml", gin.H{
"_fragment": "schedule_section_scraped",
"ScheduleDays": days,
"ScheduleYear": schedule.Year,
"ScheduleWeek": schedule.Week,
"PrevYear": prevYear,
"PrevWeek": prevWeek,
"NextYear": nextYear,
"NextWeek": nextWeek,
})
}
func (h *AnimeHandler) HandleBrowse(c *gin.Context) {
q := c.Query("q")
animeType := c.Query("type")
status := c.Query("status")
orderBy := c.Query("order_by")
sort := c.Query("sort")
sfw := c.Query("sfw") != "false"
studioID := 0
if raw := strings.TrimSpace(c.Query("studio")); raw != "" {
id, err := strconv.Atoi(raw)
if err != nil || id < 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid studio id")
return
}
studioID = id
}
var genres []int
for _, g := range c.QueryArray("genres") {
id, err := strconv.Atoi(g)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid genre id")
return
}
if id > 0 {
genres = append(genres, id)
}
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), q, animeType, status, orderBy, sort, genres, studioID, sfw, page, 24)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"browse_search_failed",
"anime",
"failed to load browse results",
map[string]any{
"q": q,
"type": animeType,
"status": status,
"order_by": orderBy,
"sort": sort,
"studio": studioID,
"sfw": sfw,
"page": page,
},
err,
)
return
}
user := server.CurrentUser(c)
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
studioName := ""
if studioID > 0 {
name, err := h.svc.GetProducerNameByID(c.Request.Context(), studioID)
if err == nil {
studioName = name
}
}
if c.GetHeader("HX-Request") == "true" && page > 1 {
c.HTML(http.StatusOK, "browse.gohtml", gin.H{
"_fragment": "anime_card_scroll",
"Animes": animes,
"NextPage": page + 1,
"HasNextPage": res.HasNextPage,
"Query": q,
"Type": animeType,
"Status": status,
"OrderBy": orderBy,
"Sort": sort,
"Genres": genres,
"Studio": studioID,
"StudioName": studioName,
"SFW": sfw,
"WatchlistMap": watchlistMap,
})
return
}
genresList, _ := h.svc.GetGenres(c.Request.Context())
browseData := gin.H{
"CurrentPath": "/browse",
"Query": q,
"Type": animeType,
"Status": status,
"OrderBy": orderBy,
"Sort": sort,
"Genres": genres,
"Studio": studioID,
"StudioName": studioName,
"SFW": sfw,
"GenresList": genresList,
"Animes": animes,
"HasNextPage": res.HasNextPage,
"NextPage": page + 1,
"User": user,
"WatchlistMap": watchlistMap,
}
if c.GetHeader("HX-Request") == "true" {
browseData["_fragment"] = "browse_content"
c.HTML(http.StatusOK, "browse.gohtml", browseData)
return
}
c.HTML(http.StatusOK, "browse.gohtml", browseData)
}
func (h *AnimeHandler) HandleAnimeDetails(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
section := c.Query("section")
if section != "" && c.GetHeader("HX-Request") == "true" {
sectionCtx, cancel := context.WithTimeout(c.Request.Context(), animeSectionTimeout)
defer cancel()
var data any
var tplName string
var err error
switch section {
case "characters":
data, err = h.svc.GetCharacters(sectionCtx, id)
tplName = "anime_characters"
case "recommendations":
data, err = h.svc.GetRecommendations(sectionCtx, id)
tplName = "anime_recommendations"
case "statistics":
data, err = h.svc.GetStatistics(sectionCtx, id)
tplName = "anime_statistics"
case "themes":
data, err = h.svc.GetThemes(sectionCtx, id)
tplName = "anime_themes"
}
if err != nil {
observability.Warn(
"anime_section_fetch_failed",
"anime",
"",
map[string]any{
"section": section,
"anime_id": id,
},
err,
)
if section == "recommendations" {
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "anime_recommendations_loading",
"AnimeID": id,
})
return
}
c.Status(http.StatusNoContent)
return
}
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": tplName,
"Items": data,
})
return
}
anime, err := h.svc.GetAnimeByID(c.Request.Context(), id)
if err != nil {
c.Status(http.StatusNotFound)
return
}
h.svc.WarmDetailSections(id)
user := server.CurrentUser(c)
status := ""
var watchlistIDs []int64
ep := 0
var cwSeconds float64
if user != nil {
entry, err := h.watchlistSvc.GetWatchListEntry(c.Request.Context(), user.ID, int64(id))
if err == nil {
status = entry.Status
watchlistIDs = []int64{entry.AnimeID}
}
cwEntry, err := h.watchlistSvc.GetContinueWatchingEntry(c.Request.Context(), user.ID, int64(id))
if err == nil && cwEntry.CurrentEpisode.Valid {
ep = int(cwEntry.CurrentEpisode.Int64)
cwSeconds = cwEntry.CurrentTimeSeconds
}
}
audioAvailability := h.animeAudioAvailability(c.Request.Context(), anime)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"Anime": anime,
"AudioAvailability": audioAvailability,
"CurrentPath": fmt.Sprintf("/anime/%d", id),
"User": user,
"Status": status,
"WatchlistIDs": watchlistIDs,
"ContinueWatchingEp": ep,
"ContinueWatchingTime": cwSeconds,
})
}
func (h *AnimeHandler) HandleHTMLWatchOrder(c *gin.Context) {
id, err := strconv.Atoi(c.Query("animeId"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
userID := server.CurrentUserID(c)
relationsCtx, cancel := context.WithTimeout(c.Request.Context(), watchOrderTimeout)
defer cancel()
relations, err := h.svc.GetRelations(relationsCtx, id)
if err != nil {
observability.Warn(
"relations_fetch_failed",
"anime",
"",
map[string]any{
"anime_id": id,
},
err,
)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order_loading",
"AnimeID": id,
})
return
}
relationAnimeIDs := make([]int64, 0, len(relations))
for _, relation := range relations {
if relation.Anime.MalID > 0 {
relationAnimeIDs = append(relationAnimeIDs, int64(relation.Anime.MalID))
}
}
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, relationAnimeIDs)
c.HTML(http.StatusOK, "anime.gohtml", gin.H{
"_fragment": "watch_order",
"Relations": relations,
"AnimeID": id,
"WatchlistMap": watchlistMap,
})
}
func (h *AnimeHandler) HandleQuickSearch(c *gin.Context) {
query := c.Query("q")
if query == "" {
c.JSON(http.StatusOK, []any{})
return
}
res, err := h.svc.SearchAdvanced(c.Request.Context(), query, "", "", "", "", nil, 0, true, 1, 5)
if err != nil {
c.JSON(http.StatusOK, []any{})
return
}
userID := server.CurrentUserID(c)
animes := wrapAnimes(res.Animes)
watchlistMap := h.watchlistMapForAnimes(c.Request.Context(), userID, animes)
type quickSearchResult struct {
ID int `json:"id"`
Title string `json:"title"`
Type string `json:"type"`
Year int `json:"year"`
Image string `json:"image"`
InWatchlist bool `json:"in_watchlist"`
}
output := make([]quickSearchResult, len(animes))
for i, anime := range animes {
output[i] = quickSearchResult{
ID: anime.MalID,
Title: anime.DisplayTitle(),
Type: anime.Type,
Year: anime.Year,
Image: anime.ImageURL(),
InWatchlist: watchlistMap[int64(anime.MalID)],
}
}
c.JSON(http.StatusOK, output)
}
func (h *AnimeHandler) HandleRandomAnime(c *gin.Context) {
ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
defer cancel()
anime, err := h.svc.GetRandomAnime(ctx)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"random_anime_fetch_failed",
"anime",
"failed to fetch random anime",
nil,
err,
)
return
}
if anime.MalID == 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadGateway, "random anime unavailable")
return
}
inWatchlist := false
userID := server.CurrentUserID(c)
if userID != "" {
watchlistMap := h.watchlistMapForIDs(c.Request.Context(), userID, []int64{int64(anime.MalID)})
inWatchlist = watchlistMap[int64(anime.MalID)]
}
c.JSON(http.StatusOK, gin.H{
"data": anime,
"in_watchlist": inWatchlist,
})
}
func (h *AnimeHandler) HandleAnimeReviews(c *gin.Context) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid anime id")
return
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, "invalid page")
return
}
if page < 1 {
page = 1
}
reviews, hasNextPage, err := h.svc.GetReviews(c.Request.Context(), id, page)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"anime_reviews_fetch_failed",
"anime",
"failed to load reviews",
map[string]any{"anime_id": id, "page": page},
err,
)
return
}
user := server.CurrentUser(c)
if c.GetHeader("HX-Request") == "true" && page > 1 {
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"_fragment": "review_cards",
"Reviews": reviews,
"NextPage": page + 1,
"HasNextPage": hasNextPage,
"AnimeID": id,
})
return
}
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"CurrentPath": fmt.Sprintf("/anime/%d/reviews", id),
"Reviews": reviews,
"NextPage": page + 1,
"HasNextPage": hasNextPage,
"AnimeID": id,
"User": user,
})
}

View File

@@ -14,7 +14,6 @@ var Module = fx.Options(
NewAnimeService, NewAnimeService,
fx.As(new(Service)), fx.As(new(Service)),
fx.As(new(domain.AnimeCatalogService)), fx.As(new(domain.AnimeCatalogService)),
fx.As(new(domain.AnimeDiscoverService)),
fx.As(new(domain.AnimeSearchService)), fx.As(new(domain.AnimeSearchService)),
fx.As(new(domain.AnimeDetailsService)), fx.As(new(domain.AnimeDetailsService)),
fx.As(new(domain.AnimePlaybackService)), fx.As(new(domain.AnimePlaybackService)),

View File

@@ -1,14 +1,19 @@
package anime package anime
import ( import (
"context"
"mal/integrations/jikan" "mal/integrations/jikan"
"mal/internal/db" "mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability"
"math" "math"
"slices" "slices"
"sort" "sort"
"strings" "strings"
"sync"
"time" "time"
"golang.org/x/sync/errgroup"
) )
const ( const (
@@ -270,6 +275,24 @@ func scoreRecommendationCandidate(
score += themeScore * forYouThemeMatchWeight score += themeScore * forYouThemeMatchWeight
score += studioScore * forYouStudioMatchWeight score += studioScore * forYouStudioMatchWeight
score += demographicScore * forYouDemographicMatchWeight score += demographicScore * forYouDemographicMatchWeight
score += recommendationCandidateScoreAdjustments(now, profile, candidate)
return recommendationCandidate{
anime: candidate,
score: score,
genreMatches: genreMatches,
themeMatches: themeMatches,
studioMatches: studioMatches,
demographicMatches: demographicMatches,
}
}
func recommendationCandidateScoreAdjustments(
now time.Time,
profile userTasteProfile,
candidate jikan.Anime,
) float64 {
var score float64
if candidate.Score > 0 { if candidate.Score > 0 {
score += min(candidate.Score/10.0, 1.0) score += min(candidate.Score/10.0, 1.0)
@@ -280,31 +303,41 @@ func scoreRecommendationCandidate(
if profile.prefersAiring && candidate.Airing { if profile.prefersAiring && candidate.Airing {
score += 0.5 score += 0.5
} }
if profile.prefersRecent && candidate.Year > 0 && now.Year()-candidate.Year <= 4 { if profile.prefersRecent && isRecentCandidate(now, candidate.Year) {
score += 0.45 score += 0.45
} }
if candidate.Year > 0 && now.Year()-candidate.Year > 15 { if isClassicCandidate(now, candidate.Year) {
score -= 0.2 score -= 0.2
} }
if candidate.Status == "Not yet aired" { if candidate.Status == "Not yet aired" {
score -= 0.35 score -= 0.35
} }
if candidate.Aired.From != "" { if isFreshRelease(now, candidate.Aired.From) {
if airedAt, err := time.Parse(time.RFC3339, candidate.Aired.From); err == nil { score += 0.3
if now.Sub(airedAt) <= forYouFreshReleaseWindow {
score += 0.3
}
}
} }
return recommendationCandidate{ return score
anime: candidate, }
score: score,
genreMatches: genreMatches, func isRecentCandidate(now time.Time, year int) bool {
themeMatches: themeMatches, return year > 0 && now.Year()-year <= 4
studioMatches: studioMatches, }
demographicMatches: demographicMatches,
func isClassicCandidate(now time.Time, year int) bool {
return year > 0 && now.Year()-year > 15
}
func isFreshRelease(now time.Time, airedFrom string) bool {
if airedFrom == "" {
return false
} }
airedAt, err := time.Parse(time.RFC3339, airedFrom)
if err != nil {
return false
}
return now.Sub(airedAt) <= forYouFreshReleaseWindow
} }
func weightedEntityMatch(weights map[int]float64, entities []jikan.NamedEntity) (int, float64) { func weightedEntityMatch(weights map[int]float64, entities []jikan.NamedEntity) (int, float64) {
@@ -501,3 +534,298 @@ func recentFeatureCounts(
} }
return counts return counts
} }
type rankedCandidate struct {
id int
collaborativeScore float64
profileSearchScore float64
anime jikan.Anime
hasAnime bool
}
type candidateStore struct {
watchlistAnimeIDs map[int]struct{}
byID map[int]rankedCandidate
mu sync.Mutex
}
func newCandidateStore(watchlist []db.GetUserWatchListRow) *candidateStore {
watchlistAnimeIDs := make(map[int]struct{}, len(watchlist))
for _, entry := range watchlist {
if entry.AnimeID <= 0 {
continue
}
watchlistAnimeIDs[int(entry.AnimeID)] = struct{}{}
}
return &candidateStore{
watchlistAnimeIDs: watchlistAnimeIDs,
byID: map[int]rankedCandidate{},
}
}
func (s *candidateStore) upsert(candidate rankedCandidate) {
if candidate.id <= 0 {
return
}
if _, exists := s.watchlistAnimeIDs[candidate.id]; exists {
return
}
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.byID[candidate.id]
if !ok {
s.byID[candidate.id] = candidate
return
}
current.collaborativeScore += candidate.collaborativeScore
current.profileSearchScore += candidate.profileSearchScore
if candidate.hasAnime {
current.anime = candidate.anime
current.hasAnime = true
}
s.byID[candidate.id] = current
}
func (s *candidateStore) ranked() []rankedCandidate {
ranked := make([]rankedCandidate, 0, len(s.byID))
for _, item := range s.byID {
ranked = append(ranked, item)
}
sort.Slice(ranked, func(i, j int) bool {
left := rankedCandidateRetrievalScore(ranked[i].collaborativeScore, ranked[i].profileSearchScore)
right := rankedCandidateRetrievalScore(ranked[j].collaborativeScore, ranked[j].profileSearchScore)
if left == right {
return ranked[i].id < ranked[j].id
}
return left > right
})
return ranked
}
func (s *animeService) GetTopPickForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouResultLimit)
}
func (s *animeService) GetTopPicksForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouFullResultLimit)
}
func (s *animeService) fetchSeedAnimes(ctx context.Context, seedPool []recommendationSeed) ([]jikan.Anime, error) {
seedAnimes := make([]jikan.Anime, len(seedPool))
var g errgroup.Group
g.SetLimit(4)
for i, seed := range seedPool {
g.Go(func() error {
anime, err := s.jikan.GetAnimeByID(ctx, seed.animeID)
if err != nil {
return err
}
seedAnimes[i] = anime
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return seedAnimes, nil
}
func (s *animeService) collectCollaborativeCandidates(ctx context.Context, seedPool []recommendationSeed, store *candidateStore) error {
var g errgroup.Group
g.SetLimit(4)
for _, seed := range seedPool {
g.Go(func() error {
recs, err := s.jikan.GetAnimeRecommendations(ctx, seed.animeID)
if err != nil {
return err
}
for i, rec := range recs {
if i >= forYouMaxRecommendations {
break
}
id := rec.Entry.MalID
if id <= 0 || id == seed.animeID {
continue
}
store.upsert(rankedCandidate{
id: id,
collaborativeScore: float64(rec.Votes) * seed.weight,
})
}
return nil
})
}
return g.Wait()
}
func (s *animeService) collectProfileSearchCandidates(ctx context.Context, profile userTasteProfile, store *candidateStore) error {
queries := buildProfileSearchQueries(profile)
var g errgroup.Group
g.SetLimit(3)
for _, query := range queries {
g.Go(func() error {
res, err := s.jikan.SearchAdvanced(
ctx,
"",
"",
"",
"score",
"desc",
query.genreIDs,
query.studioID,
true,
1,
forYouProfileSearchLimit,
)
if err != nil {
observability.Warn(
"top_pick_profile_search_failed",
"anime",
"",
map[string]any{
"genres": query.genreIDs,
"studio_id": query.studioID,
},
err,
)
return nil
}
for i, anime := range res.Animes {
if anime.MalID <= 0 {
continue
}
store.upsert(rankedCandidate{
id: anime.MalID,
profileSearchScore: query.weight * profileSearchRankWeight(i),
anime: anime,
hasAnime: true,
})
}
return nil
})
}
return g.Wait()
}
func (s *animeService) scoreRankedCandidates(
ctx context.Context,
now time.Time,
profile userTasteProfile,
ranked []rankedCandidate,
) ([]recommendationCandidate, error) {
limit := min(len(ranked), forYouCandidateFetchLimit)
candidates := make([]recommendationCandidate, 0, limit)
var candidatesMu sync.Mutex
var g errgroup.Group
g.SetLimit(6)
for i := 0; i < limit; i++ {
item := ranked[i]
g.Go(func() error {
anime := item.anime
if !item.hasAnime || !hasTasteMetadata(anime) {
fetchedAnime, err := s.jikan.GetAnimeByID(ctx, item.id)
if err != nil {
observability.Warn(
"recommendation_anime_fetch_failed",
"anime",
"",
map[string]any{"anime_id": item.id},
err,
)
return nil
}
anime = fetchedAnime
}
candidate := scoreRecommendationCandidate(
now,
profile,
anime,
item.collaborativeScore,
item.profileSearchScore,
)
candidatesMu.Lock()
candidates = append(candidates, candidate)
candidatesMu.Unlock()
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].score == candidates[j].score {
return candidates[i].anime.MalID < candidates[j].anime.MalID
}
return candidates[i].score > candidates[j].score
})
return candidates, nil
}
func (s *animeService) getTopPicksForYou(
ctx context.Context,
userID string,
resultLimit int,
) (domain.CatalogSectionData, error) {
if strings.TrimSpace(userID) == "" {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return domain.CatalogSectionData{}, err
}
now := time.Now()
seedPool := buildRecommendationSeeds(now, watchlist)
if len(seedPool) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
seedAnimes, err := s.fetchSeedAnimes(ctx, seedPool)
if err != nil {
return domain.CatalogSectionData{}, err
}
profile := buildTasteProfile(now, seedPool, seedAnimes)
store := newCandidateStore(watchlist)
if err := s.collectCollaborativeCandidates(ctx, seedPool, store); err != nil {
return domain.CatalogSectionData{}, err
}
if err := s.collectProfileSearchCandidates(ctx, profile, store); err != nil {
return domain.CatalogSectionData{}, err
}
ranked := store.ranked()
if len(ranked) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
candidates, err := s.scoreRankedCandidates(ctx, now, profile, ranked)
if err != nil {
return domain.CatalogSectionData{}, err
}
return domain.CatalogSectionData{
Animes: rerankRecommendationCandidates(candidates, resultLimit),
}, nil
}

View File

@@ -0,0 +1,76 @@
package anime
import (
"fmt"
"mal/internal/server"
"net/http"
"strconv"
"github.com/gin-gonic/gin"
)
type reviewsQuery struct {
animeID int
page int
}
func parseReviewsQuery(c *gin.Context) (reviewsQuery, error) {
id, err := strconv.Atoi(c.Param("id"))
if err != nil || id <= 0 {
return reviewsQuery{}, fmt.Errorf("invalid anime id")
}
page, err := strconv.Atoi(c.DefaultQuery("page", "1"))
if err != nil {
return reviewsQuery{}, fmt.Errorf("invalid page")
}
if page < 1 {
page = 1
}
return reviewsQuery{animeID: id, page: page}, nil
}
func (h *AnimeHandler) HandleAnimeReviews(c *gin.Context) {
query, err := parseReviewsQuery(c)
if err != nil {
server.RespondHTMLOrJSONError(c, http.StatusBadRequest, err.Error())
return
}
reviews, hasNextPage, err := h.svc.GetReviews(c.Request.Context(), query.animeID, query.page)
if err != nil {
server.RespondError(
c,
http.StatusInternalServerError,
"anime_reviews_fetch_failed",
"anime",
"failed to load reviews",
map[string]any{"anime_id": query.animeID, "page": query.page},
err,
)
return
}
user := server.CurrentUser(c)
if c.GetHeader("HX-Request") == "true" && query.page > 1 {
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"_fragment": "review_cards",
"Reviews": reviews,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"AnimeID": query.animeID,
})
return
}
c.HTML(http.StatusOK, "reviews.gohtml", gin.H{
"CurrentPath": fmt.Sprintf("/anime/%d/reviews", query.animeID),
"Reviews": reviews,
"NextPage": query.page + 1,
"HasNextPage": hasNextPage,
"AnimeID": query.animeID,
"User": user,
})
}

View File

@@ -45,9 +45,9 @@ func (h *AnimeHandler) getCachedAnimeScheduleWeek(ctx context.Context, year int,
cacheKey := fmt.Sprintf("%d-%02d-%s", year, week, timezone) cacheKey := fmt.Sprintf("%d-%02d-%s", year, week, timezone)
const ttl = 10 * time.Minute const ttl = 10 * time.Minute
h.scheduleCacheMu.Lock() h.Lock()
cached, ok := h.scheduleCache[cacheKey] cached, ok := h.scheduleCache[cacheKey]
h.scheduleCacheMu.Unlock() h.Unlock()
if ok && time.Since(cached.fetchedAt) < ttl { if ok && time.Since(cached.fetchedAt) < ttl {
return cached.value, nil return cached.value, nil
@@ -58,9 +58,9 @@ func (h *AnimeHandler) getCachedAnimeScheduleWeek(ctx context.Context, year int,
return animeschedule.WeekSchedule{}, err return animeschedule.WeekSchedule{}, err
} }
h.scheduleCacheMu.Lock() h.Lock()
h.scheduleCache[cacheKey] = cachedWeekSchedule{fetchedAt: time.Now(), value: value} h.scheduleCache[cacheKey] = cachedWeekSchedule{fetchedAt: time.Now(), value: value}
h.scheduleCacheMu.Unlock() h.Unlock()
return value, nil return value, nil
} }

View File

@@ -1,17 +1,13 @@
// Package anime provides anime catalog, discovery, search, and details services. // Package anime provides anime catalog, search, and details services.
package anime package anime
import ( import (
"context" "context"
"errors"
"mal/integrations/jikan" "mal/integrations/jikan"
"mal/internal/db" "mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability"
"math/rand" "math/rand"
"sort"
"strings" "strings"
"sync"
"time" "time"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
@@ -76,355 +72,6 @@ func (s *animeService) GetCatalogSection(ctx context.Context, userID string, sec
}, nil }, nil
} }
func (s *animeService) GetDiscoverSection(ctx context.Context, userID string, section string) (domain.DiscoverSectionData, error) {
var res jikan.TopAnimeResult
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
switch section {
case "Trending":
res, err = s.jikan.GetSeasonsNow(gCtx, 1)
case "Upcoming":
res, err = s.jikan.GetSeasonsUpcoming(gCtx, 1)
case "Top":
res, err = s.jikan.GetTopAnime(gCtx, 1)
}
return err
})
if err := g.Wait(); err != nil {
return domain.DiscoverSectionData{}, err
}
animes := wrapAnimes(res.Animes)
if len(animes) > 8 {
animes = animes[:8]
}
return domain.DiscoverSectionData{
Animes: animes,
}, nil
}
func (s *animeService) GetTopPickForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouResultLimit)
}
func (s *animeService) GetTopPicksForYou(ctx context.Context, userID string) (domain.CatalogSectionData, error) {
return s.getTopPicksForYou(ctx, userID, forYouFullResultLimit)
}
func (s *animeService) getTopPicksForYou(
ctx context.Context,
userID string,
resultLimit int,
) (domain.CatalogSectionData, error) {
if strings.TrimSpace(userID) == "" {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return domain.CatalogSectionData{}, err
}
now := time.Now()
seedPool := buildRecommendationSeeds(now, watchlist)
if len(seedPool) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
type rankedCandidate struct {
id int
collaborativeScore float64
profileSearchScore float64
anime jikan.Anime
hasAnime bool
}
watchlistAnimeIDs := make(map[int]struct{}, len(watchlist))
for _, entry := range watchlist {
if entry.AnimeID <= 0 {
continue
}
watchlistAnimeIDs[int(entry.AnimeID)] = struct{}{}
}
candidatesByID := map[int]rankedCandidate{}
var candidatesByIDMu sync.Mutex
upsertCandidate := func(candidate rankedCandidate) {
if candidate.id <= 0 {
return
}
if _, exists := watchlistAnimeIDs[candidate.id]; exists {
return
}
candidatesByIDMu.Lock()
defer candidatesByIDMu.Unlock()
current, ok := candidatesByID[candidate.id]
if !ok {
candidatesByID[candidate.id] = candidate
return
}
current.collaborativeScore += candidate.collaborativeScore
current.profileSearchScore += candidate.profileSearchScore
if candidate.hasAnime {
current.anime = candidate.anime
current.hasAnime = true
}
candidatesByID[candidate.id] = current
}
seedAnimes := make([]jikan.Anime, len(seedPool))
var seedFetchGroup errgroup.Group
seedFetchGroup.SetLimit(4)
for i, seed := range seedPool {
seedFetchGroup.Go(func() error {
anime, fetchErr := s.jikan.GetAnimeByID(ctx, seed.animeID)
if fetchErr != nil {
return fetchErr
}
seedAnimes[i] = anime
return nil
})
}
if err := seedFetchGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
profile := buildTasteProfile(now, seedPool, seedAnimes)
var recommendationGroup errgroup.Group
recommendationGroup.SetLimit(4)
for _, seed := range seedPool {
recommendationGroup.Go(func() error {
recs, recErr := s.jikan.GetAnimeRecommendations(ctx, seed.animeID)
if recErr != nil {
return recErr
}
for i, rec := range recs {
if i >= forYouMaxRecommendations {
break
}
id := rec.Entry.MalID
if id <= 0 {
continue
}
if id == seed.animeID {
continue
}
upsertCandidate(rankedCandidate{
id: id,
collaborativeScore: float64(rec.Votes) * seed.weight,
})
}
return nil
})
}
if err := recommendationGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
profileQueries := buildProfileSearchQueries(profile)
var profileSearchGroup errgroup.Group
profileSearchGroup.SetLimit(3)
for _, query := range profileQueries {
profileSearchGroup.Go(func() error {
res, searchErr := s.jikan.SearchAdvanced(
ctx,
"",
"",
"",
"score",
"desc",
query.genreIDs,
query.studioID,
true,
1,
forYouProfileSearchLimit,
)
if searchErr != nil {
observability.Warn(
"top_pick_profile_search_failed",
"anime",
"",
map[string]any{
"genres": query.genreIDs,
"studio_id": query.studioID,
},
searchErr,
)
return nil
}
for i, anime := range res.Animes {
if anime.MalID <= 0 {
continue
}
upsertCandidate(rankedCandidate{
id: anime.MalID,
profileSearchScore: query.weight * profileSearchRankWeight(i),
anime: anime,
hasAnime: true,
})
}
return nil
})
}
if err := profileSearchGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
if len(candidatesByID) == 0 {
return domain.CatalogSectionData{Animes: []domain.Anime{}}, nil
}
rankedIDs := make([]rankedCandidate, 0, len(candidatesByID))
for _, item := range candidatesByID {
rankedIDs = append(rankedIDs, item)
}
sort.Slice(rankedIDs, func(i, j int) bool {
left := rankedCandidateRetrievalScore(rankedIDs[i].collaborativeScore, rankedIDs[i].profileSearchScore)
right := rankedCandidateRetrievalScore(rankedIDs[j].collaborativeScore, rankedIDs[j].profileSearchScore)
if left == right {
return rankedIDs[i].id < rankedIDs[j].id
}
return left > right
})
limit := min(len(rankedIDs), forYouCandidateFetchLimit)
candidates := make([]recommendationCandidate, 0, limit)
var candidatesMu sync.Mutex
var detailGroup errgroup.Group
detailGroup.SetLimit(6)
for i := 0; i < limit; i++ {
item := rankedIDs[i]
detailGroup.Go(func() error {
anime := item.anime
if !item.hasAnime || !hasTasteMetadata(anime) {
fetchedAnime, fetchErr := s.jikan.GetAnimeByID(ctx, item.id)
if fetchErr != nil {
observability.Warn(
"recommendation_anime_fetch_failed",
"anime",
"",
map[string]any{"anime_id": item.id},
fetchErr,
)
return nil
}
anime = fetchedAnime
}
candidate := scoreRecommendationCandidate(
now,
profile,
anime,
item.collaborativeScore,
item.profileSearchScore,
)
candidatesMu.Lock()
candidates = append(candidates, candidate)
candidatesMu.Unlock()
return nil
})
}
if err := detailGroup.Wait(); err != nil {
return domain.CatalogSectionData{}, err
}
sort.Slice(candidates, func(i, j int) bool {
if candidates[i].score == candidates[j].score {
return candidates[i].anime.MalID < candidates[j].anime.MalID
}
return candidates[i].score > candidates[j].score
})
return domain.CatalogSectionData{
Animes: rerankRecommendationCandidates(candidates, resultLimit),
}, nil
}
func (s *animeService) GetAiringSchedule(ctx context.Context, userID string) ([]domain.Anime, error) {
if strings.TrimSpace(userID) == "" {
return []domain.Anime{}, nil
}
watchlist, err := s.repo.GetUserWatchList(ctx, userID)
if err != nil {
return nil, err
}
ids := make([]int, 0, 50)
for _, entry := range watchlist {
status := strings.TrimSpace(entry.Status)
if status != "watching" && status != "plan_to_watch" {
continue
}
if !entry.Airing.Valid || !entry.Airing.Bool {
continue
}
if entry.AnimeID <= 0 {
continue
}
ids = append(ids, int(entry.AnimeID))
if len(ids) >= 50 {
break
}
}
if len(ids) == 0 {
return []domain.Anime{}, nil
}
animes := make([]domain.Anime, 0, len(ids))
var g errgroup.Group
g.SetLimit(6)
var mu sync.Mutex
for _, id := range ids {
g.Go(func() error {
anime, fetchErr := s.jikan.GetAnimeByID(ctx, id)
if fetchErr != nil {
return fetchErr
}
mu.Lock()
animes = append(animes, domain.Anime{Anime: anime})
mu.Unlock()
return nil
})
}
if err := g.Wait(); err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return nil, err
}
observability.Warn(
"schedule_partial_fetch_failed",
"anime",
"",
map[string]any{"user_id": userID, "count": len(ids)},
err,
)
return animes, nil
}
return animes, nil
}
func (s *animeService) GetAnimeByID(ctx context.Context, id int) (domain.Anime, error) { func (s *animeService) GetAnimeByID(ctx context.Context, id int) (domain.Anime, error) {
anime, err := s.jikan.GetAnimeByID(ctx, id) anime, err := s.jikan.GetAnimeByID(ctx, id)
if err != nil { if err != nil {
@@ -525,8 +172,8 @@ func (s *animeService) GetRecommendations(ctx context.Context, id int) ([]domain
return out, nil return out, nil
} }
func (s *animeService) GetRelations(ctx context.Context, id int) ([]jikan.RelationEntry, error) { func (s *animeService) GetRelations(ctx context.Context, id int, mode jikan.WatchOrderMode) ([]jikan.RelationEntry, error) {
return s.jikan.GetFullRelations(ctx, id) return s.jikan.GetFullRelations(ctx, id, mode)
} }
func (s *animeService) WarmDetailSections(id int) { func (s *animeService) WarmDetailSections(id int) {

View File

@@ -10,6 +10,7 @@ import (
"mal/internal/config" "mal/internal/config"
"mal/internal/database" "mal/internal/database"
"mal/internal/episodes" "mal/internal/episodes"
"mal/internal/observability"
"mal/internal/playback" "mal/internal/playback"
"mal/internal/server" "mal/internal/server"
"mal/internal/watchlist" "mal/internal/watchlist"
@@ -22,6 +23,7 @@ import (
func NewApp() *fx.App { func NewApp() *fx.App {
return fx.New( return fx.New(
fx.WithLogger(observability.NewFxLogger),
config.Module, config.Module,
database.Module, database.Module,
audit.Module, audit.Module,

View File

@@ -2,6 +2,7 @@ package audit_test
import ( import (
"context" "context"
"database/sql"
"encoding/json" "encoding/json"
"os" "os"
"testing" "testing"
@@ -13,29 +14,9 @@ import (
) )
func TestRecordInsertsAuditLog(t *testing.T) { func TestRecordInsertsAuditLog(t *testing.T) {
tmp, err := os.CreateTemp("", "mal-audit-*.db") sqlDB := openTestDB(t)
if err != nil { svc := audit.NewAuditService(db.New(sqlDB))
t.Fatalf("CreateTemp: %v", err) insertTestUser(t, sqlDB, "user-1")
}
_ = tmp.Close()
t.Cleanup(func() { _ = os.Remove(tmp.Name()) })
sqlDB, err := db.Open(tmp.Name())
if err != nil {
t.Fatalf("db.Open: %v", err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
if err := database.RunMigrations(sqlDB); err != nil {
t.Fatalf("RunMigrations: %v", err)
}
queries := db.New(sqlDB)
svc := audit.NewAuditService(queries)
if _, err := sqlDB.Exec("INSERT INTO user (id, username, password_hash) VALUES (?, ?, ?)", "user-1", "test", "hash"); err != nil {
t.Fatalf("insert user: %v", err)
}
ctx := audit.WithRequestInfo(context.Background(), "127.0.0.1", "unit-test") ctx := audit.WithRequestInfo(context.Background(), "127.0.0.1", "unit-test")
metadata, err := json.Marshal(struct { metadata, err := json.Marshal(struct {
@@ -55,7 +36,54 @@ func TestRecordInsertsAuditLog(t *testing.T) {
t.Fatalf("Record: %v", err) t.Fatalf("Record: %v", err)
} }
rows, err := sqlDB.Query("SELECT action, resource_type, resource_id, ip, user_agent, metadata_json FROM audit_log WHERE user_id = ?", "user-1") auditRow := queryAuditRow(t, sqlDB, "user-1")
assertAuditRow(t, auditRow)
}
type auditRow struct {
action string
resourceType string
resourceID string
ip string
userAgent string
metadataJSON string
}
func openTestDB(t *testing.T) *sql.DB {
t.Helper()
tmp, err := os.CreateTemp("", "mal-audit-*.db")
if err != nil {
t.Fatalf("CreateTemp: %v", err)
}
_ = tmp.Close()
t.Cleanup(func() { _ = os.Remove(tmp.Name()) })
sqlDB, err := db.Open(tmp.Name())
if err != nil {
t.Fatalf("db.Open: %v", err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
if err := database.RunMigrations(sqlDB); err != nil {
t.Fatalf("RunMigrations: %v", err)
}
return sqlDB
}
func insertTestUser(t *testing.T, sqlDB *sql.DB, userID string) {
t.Helper()
if _, err := sqlDB.ExecContext(context.Background(), "INSERT INTO user (id, username, password_hash) VALUES (?, ?, ?)", userID, "test", "hash"); err != nil {
t.Fatalf("insert user: %v", err)
}
}
func queryAuditRow(t *testing.T, sqlDB *sql.DB, userID string) auditRow {
t.Helper()
rows, err := sqlDB.QueryContext(context.Background(), "SELECT action, resource_type, resource_id, ip, user_agent, metadata_json FROM audit_log WHERE user_id = ?", userID)
if err != nil { if err != nil {
t.Fatalf("Query: %v", err) t.Fatalf("Query: %v", err)
} }
@@ -65,18 +93,24 @@ func TestRecordInsertsAuditLog(t *testing.T) {
t.Fatalf("expected audit row") t.Fatalf("expected audit row")
} }
var action, resourceType, resourceID, ip, userAgent, metadataJSON string var row auditRow
if err := rows.Scan(&action, &resourceType, &resourceID, &ip, &userAgent, &metadataJSON); err != nil { if err := rows.Scan(&row.action, &row.resourceType, &row.resourceID, &row.ip, &row.userAgent, &row.metadataJSON); err != nil {
t.Fatalf("Scan: %v", err) t.Fatalf("Scan: %v", err)
} }
if action != "test_action" || resourceType != "thing" || resourceID != "123" { return row
t.Fatalf("unexpected row action=%q resourceType=%q resourceID=%q", action, resourceType, resourceID) }
func assertAuditRow(t *testing.T, row auditRow) {
t.Helper()
if row.action != "test_action" || row.resourceType != "thing" || row.resourceID != "123" {
t.Fatalf("unexpected row action=%q resourceType=%q resourceID=%q", row.action, row.resourceType, row.resourceID)
} }
if ip != "127.0.0.1" || userAgent != "unit-test" { if row.ip != "127.0.0.1" || row.userAgent != "unit-test" {
t.Fatalf("unexpected request info ip=%q userAgent=%q", ip, userAgent) t.Fatalf("unexpected request info ip=%q userAgent=%q", row.ip, row.userAgent)
} }
if metadataJSON == "" || metadataJSON == "null" { if row.metadataJSON == "" || row.metadataJSON == "null" {
t.Fatalf("expected metadata_json, got %q", metadataJSON) t.Fatalf("expected metadata_json, got %q", row.metadataJSON)
} }
} }

View File

@@ -49,6 +49,33 @@ func isPublicRequest(method string, path string) bool {
return false return false
} }
func authenticateAPIRequest(c *gin.Context, svc domain.AuthService) (*domain.User, string, bool, error) {
authHeader := strings.TrimSpace(c.GetHeader("Authorization"))
if strings.HasPrefix(strings.ToLower(authHeader), "bearer ") {
token := strings.TrimSpace(authHeader[7:])
user, err := svc.ValidateAPIToken(c.Request.Context(), token)
return user, "", false, err
}
sessionID, err := c.Cookie("session_id")
if err != nil {
return nil, "", false, err
}
user, err := svc.ValidateSession(c.Request.Context(), sessionID)
return user, sessionID, true, err
}
func authenticatePageRequest(c *gin.Context, svc domain.AuthService) (*domain.User, string, error) {
sessionID, err := c.Cookie("session_id")
if err != nil {
return nil, "", err
}
user, err := svc.ValidateSession(c.Request.Context(), sessionID)
return user, sessionID, err
}
func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc { func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
return func(c *gin.Context) { return func(c *gin.Context) {
path := c.Request.URL.Path path := c.Request.URL.Path
@@ -65,18 +92,7 @@ func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
// API routes can authenticate via Bearer token OR cookie session. // API routes can authenticate via Bearer token OR cookie session.
if strings.HasPrefix(path, "/api/") { if strings.HasPrefix(path, "/api/") {
authHeader := strings.TrimSpace(c.GetHeader("Authorization")) user, sessionID, usesCookieSession, err = authenticateAPIRequest(c, svc)
if strings.HasPrefix(strings.ToLower(authHeader), "bearer ") {
token := strings.TrimSpace(authHeader[7:])
user, err = svc.ValidateAPIToken(c.Request.Context(), token)
} else if cookieSessionID, cookieErr := c.Cookie("session_id"); cookieErr == nil {
sessionID = cookieSessionID
usesCookieSession = true
user, err = svc.ValidateSession(c.Request.Context(), sessionID)
} else {
err = cookieErr
}
if err != nil || user == nil { if err != nil || user == nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "Unauthorized"}) c.JSON(http.StatusUnauthorized, gin.H{"error": "Unauthorized"})
c.Abort() c.Abort()
@@ -84,16 +100,8 @@ func AuthMiddleware(svc domain.AuthService) gin.HandlerFunc {
} }
} else { } else {
// Non-API routes only use cookie sessions and redirect to /login. // Non-API routes only use cookie sessions and redirect to /login.
cookieSessionID, cookieErr := c.Cookie("session_id") user, sessionID, err = authenticatePageRequest(c, svc)
if cookieErr != nil {
c.Redirect(http.StatusSeeOther, "/login")
c.Abort()
return
}
sessionID = cookieSessionID
usesCookieSession = true usesCookieSession = true
user, err = svc.ValidateSession(c.Request.Context(), sessionID)
if err != nil || user == nil { if err != nil || user == nil {
c.Redirect(http.StatusSeeOther, "/login") c.Redirect(http.StatusSeeOther, "/login")
c.Abort() c.Abort()

View File

@@ -1,10 +1,8 @@
package auth package auth
import ( import (
"mal/internal/domain"
"mal/internal/server" "mal/internal/server"
"github.com/gin-gonic/gin"
"go.uber.org/fx" "go.uber.org/fx"
) )
@@ -13,9 +11,7 @@ var Module = fx.Options(
NewAuthRepository, NewAuthRepository,
NewAuthService, NewAuthService,
NewAuthHandler, NewAuthHandler,
func(svc domain.AuthService) gin.HandlerFunc { AuthMiddleware,
return AuthMiddleware(svc)
},
), ),
fx.Provide( fx.Provide(
server.AsRouteRegister(func(h *AuthHandler) server.RouteRegister { server.AsRouteRegister(func(h *AuthHandler) server.RouteRegister {

View File

@@ -6,6 +6,7 @@ import (
) )
func DefaultAvatarURL(username string) string { func DefaultAvatarURL(username string) string {
seed := url.QueryEscape(strings.TrimSpace(username)) params := url.Values{}
return "https://api.dicebear.com/9.x/dylan/svg?seed=" + seed params.Set("seed", strings.TrimSpace(username))
return "https://api.dicebear.com/9.x/dylan/svg?" + params.Encode()
} }

View File

@@ -38,6 +38,7 @@ func ProvideQueries(sqlDB *sql.DB) *db.Queries {
func RunMigrations(sqlDB *sql.DB) error { func RunMigrations(sqlDB *sql.DB) error {
goose.SetBaseFS(migrationsFS) goose.SetBaseFS(migrationsFS)
goose.SetLogger(goose.NopLogger())
if err := goose.SetDialect("sqlite3"); err != nil { if err := goose.SetDialect("sqlite3"); err != nil {
return fmt.Errorf("failed to set goose dialect: %w", err) return fmt.Errorf("failed to set goose dialect: %w", err)
@@ -48,6 +49,13 @@ func RunMigrations(sqlDB *sql.DB) error {
return fmt.Errorf("failed to run migrations: %w", err) return fmt.Errorf("failed to run migrations: %w", err)
} }
version, err := goose.GetDBVersion(sqlDB)
if err != nil {
return fmt.Errorf("failed to get database migration version: %w", err)
}
observability.Info("db_migrations_complete", "database", "", map[string]any{"version": version})
return nil return nil
} }
func RunMigrationsAndFixes(sqlDB *sql.DB) error { func RunMigrationsAndFixes(sqlDB *sql.DB) error {

View File

@@ -1,6 +1,7 @@
package database package database
import ( import (
"context"
"database/sql" "database/sql"
"testing" "testing"
@@ -28,7 +29,7 @@ func TestRunMigrationsCreatesHotPathIndexes(t *testing.T) {
} { } {
t.Run(indexName, func(t *testing.T) { t.Run(indexName, func(t *testing.T) {
var count int var count int
err := sqlDB.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type = 'index' AND name = ?`, indexName).Scan(&count) err := sqlDB.QueryRowContext(context.Background(), `SELECT COUNT(*) FROM sqlite_master WHERE type = 'index' AND name = ?`, indexName).Scan(&count)
if err != nil { if err != nil {
t.Fatalf("query index: %v", err) t.Fatalf("query index: %v", err)
} }

View File

@@ -10,72 +10,82 @@ import (
"mal/internal/observability" "mal/internal/observability"
) )
type animeDurationRow struct {
id int64
titleOriginal string
}
func init() { func init() {
Register(Fix{ Register(Fix{
ID: "20260608_backfill_anime_duration_seconds", ID: "20260608_backfill_anime_duration_seconds",
Apply: func(ctx context.Context, sqlDB *sql.DB) error { Apply: applyAnimeDurationSecondsBackfill,
rows, err := sqlDB.QueryContext(ctx, ` })
}
func applyAnimeDurationSecondsBackfill(ctx context.Context, sqlDB *sql.DB) error {
toUpdate, err := listAnimeMissingDurationSeconds(ctx, sqlDB)
if err != nil {
return err
}
client := jikan.NewClient(config.Config{}, db.New(sqlDB), observability.NewMetrics())
for _, row := range toUpdate {
anime, err := client.GetAnimeByID(ctx, int(row.id))
if err != nil {
return fmt.Errorf("fetch anime %d for duration backfill: %w", row.id, err)
}
durationSeconds := anime.DurationSeconds()
if durationSeconds <= 0 {
continue
}
if _, err := sqlDB.ExecContext(
ctx,
`UPDATE anime SET duration_seconds = ? WHERE id = ? AND duration_seconds IS NULL`,
durationSeconds,
row.id,
); err != nil {
return fmt.Errorf("update anime %d duration_seconds: %w", row.id, err)
}
}
return nil
}
func listAnimeMissingDurationSeconds(ctx context.Context, sqlDB *sql.DB) ([]animeDurationRow, error) {
rows, err := sqlDB.QueryContext(ctx, `
SELECT id, title_original, title_english, title_japanese, image_url, airing SELECT id, title_original, title_english, title_japanese, image_url, airing
FROM anime FROM anime
WHERE duration_seconds IS NULL; WHERE duration_seconds IS NULL;
`) `)
if err != nil { if err != nil {
return fmt.Errorf("query anime rows missing duration_seconds: %w", err) return nil, fmt.Errorf("query anime rows missing duration_seconds: %w", err)
} }
defer func() { _ = rows.Close() }() defer func() { _ = rows.Close() }()
client := jikan.NewClient(config.Config{}, db.New(sqlDB), observability.NewMetrics()) var toUpdate []animeDurationRow
for rows.Next() {
var row animeDurationRow
var titleEnglish sql.NullString
var titleJapanese sql.NullString
var imageURL string
var airing sql.NullBool
if err := rows.Scan(
&row.id,
&row.titleOriginal,
&titleEnglish,
&titleJapanese,
&imageURL,
&airing,
); err != nil {
return nil, fmt.Errorf("scan anime row missing duration_seconds: %w", err)
}
toUpdate = append(toUpdate, row)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate anime rows missing duration_seconds: %w", err)
}
type animeRow struct { return toUpdate, nil
id int64
titleOriginal string
}
var toUpdate []animeRow
for rows.Next() {
var row animeRow
var titleEnglish sql.NullString
var titleJapanese sql.NullString
var imageURL string
var airing sql.NullBool
if err := rows.Scan(
&row.id,
&row.titleOriginal,
&titleEnglish,
&titleJapanese,
&imageURL,
&airing,
); err != nil {
return fmt.Errorf("scan anime row missing duration_seconds: %w", err)
}
toUpdate = append(toUpdate, row)
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate anime rows missing duration_seconds: %w", err)
}
for _, row := range toUpdate {
anime, err := client.GetAnimeByID(ctx, int(row.id))
if err != nil {
return fmt.Errorf("fetch anime %d for duration backfill: %w", row.id, err)
}
durationSeconds := anime.DurationSeconds()
if durationSeconds <= 0 {
continue
}
if _, err := sqlDB.ExecContext(
ctx,
`UPDATE anime SET duration_seconds = ? WHERE id = ? AND duration_seconds IS NULL`,
durationSeconds,
row.id,
); err != nil {
return fmt.Errorf("update anime %d duration_seconds: %w", row.id, err)
}
}
return nil
},
})
} }

View File

@@ -9,9 +9,7 @@ func (q *Queries) GetCommandPaletteContinueWatching(ctx context.Context, userID
if userID == "" { if userID == "" {
return nil, nil return nil, nil
} }
if limit <= 0 { limit = commandPaletteLimit(limit)
limit = 5
}
needle, pattern := commandPalettePattern(query) needle, pattern := commandPalettePattern(query)
rows, err := q.db.QueryContext(ctx, ` rows, err := q.db.QueryContext(ctx, `
@@ -48,22 +46,8 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
items := make([]GetContinueWatchingEntriesRow, 0, int(limit)) items := make([]GetContinueWatchingEntriesRow, 0, int(limit))
for rows.Next() { for rows.Next() {
var item GetContinueWatchingEntriesRow item, err := scanContinueWatchingEntry(rows)
if err := rows.Scan( if err != nil {
&item.ID,
&item.UserID,
&item.AnimeID,
&item.CurrentEpisode,
&item.CurrentTimeSeconds,
&item.DurationSeconds,
&item.CreatedAt,
&item.UpdatedAt,
&item.TitleOriginal,
&item.TitleEnglish,
&item.TitleJapanese,
&item.ImageUrl,
&item.AnimeDurationSeconds,
); err != nil {
return nil, err return nil, err
} }
items = append(items, item) items = append(items, item)
@@ -75,13 +59,31 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
return items, nil return items, nil
} }
func scanContinueWatchingEntry(rows scanner) (GetContinueWatchingEntriesRow, error) {
var item GetContinueWatchingEntriesRow
err := rows.Scan(
&item.ID,
&item.UserID,
&item.AnimeID,
&item.CurrentEpisode,
&item.CurrentTimeSeconds,
&item.DurationSeconds,
&item.CreatedAt,
&item.UpdatedAt,
&item.TitleOriginal,
&item.TitleEnglish,
&item.TitleJapanese,
&item.ImageUrl,
&item.AnimeDurationSeconds,
)
return item, err
}
func (q *Queries) GetCommandPaletteWatchlist(ctx context.Context, userID string, query string, limit int64) ([]GetUserWatchListRow, error) { func (q *Queries) GetCommandPaletteWatchlist(ctx context.Context, userID string, query string, limit int64) ([]GetUserWatchListRow, error) {
if userID == "" { if userID == "" {
return nil, nil return nil, nil
} }
if limit <= 0 { limit = commandPaletteLimit(limit)
limit = 5
}
needle, pattern := commandPalettePattern(query) needle, pattern := commandPalettePattern(query)
rows, err := q.db.QueryContext(ctx, ` rows, err := q.db.QueryContext(ctx, `
@@ -126,23 +128,8 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
items := make([]GetUserWatchListRow, 0, int(limit)) items := make([]GetUserWatchListRow, 0, int(limit))
for rows.Next() { for rows.Next() {
var item GetUserWatchListRow item, err := scanWatchListEntry(rows)
if err := rows.Scan( if err != nil {
&item.ID,
&item.UserID,
&item.AnimeID,
&item.Status,
&item.CreatedAt,
&item.UpdatedAt,
&item.CurrentEpisode,
&item.LastEpisodeAt,
&item.CurrentTimeSeconds,
&item.TitleOriginal,
&item.TitleEnglish,
&item.TitleJapanese,
&item.ImageUrl,
&item.Airing,
); err != nil {
return nil, err return nil, err
} }
items = append(items, item) items = append(items, item)
@@ -154,7 +141,40 @@ LIMIT ?`, userID, needle, pattern, pattern, pattern, pattern, limit)
return items, nil return items, nil
} }
func scanWatchListEntry(rows scanner) (GetUserWatchListRow, error) {
var item GetUserWatchListRow
err := rows.Scan(
&item.ID,
&item.UserID,
&item.AnimeID,
&item.Status,
&item.CreatedAt,
&item.UpdatedAt,
&item.CurrentEpisode,
&item.LastEpisodeAt,
&item.CurrentTimeSeconds,
&item.TitleOriginal,
&item.TitleEnglish,
&item.TitleJapanese,
&item.ImageUrl,
&item.Airing,
)
return item, err
}
func commandPalettePattern(query string) (string, string) { func commandPalettePattern(query string) (string, string) {
needle := strings.ToLower(strings.TrimSpace(query)) needle := strings.ToLower(strings.TrimSpace(query))
return needle, "%" + needle + "%" return needle, "%" + needle + "%"
} }
func commandPaletteLimit(limit int64) int64 {
if limit <= 0 {
return 5
}
return limit
}
type scanner interface {
Scan(dest ...interface{}) error
}

View File

@@ -60,7 +60,7 @@ func openCommandPaletteTestDB(t *testing.T) *sql.DB {
} }
t.Cleanup(func() { _ = sqlDB.Close() }) t.Cleanup(func() { _ = sqlDB.Close() })
_, err = sqlDB.Exec(` _, err = sqlDB.ExecContext(context.Background(), `
CREATE TABLE anime ( CREATE TABLE anime (
id INTEGER PRIMARY KEY, id INTEGER PRIMARY KEY,
title_original TEXT NOT NULL, title_original TEXT NOT NULL,

View File

@@ -11,9 +11,15 @@ func NullStringOr(n sql.NullString, fallback string) string {
return fallback return fallback
} }
// DisplayTitle returns the English title, falling back to Japanese then original // DisplayTitle returns the English title, falling back to original then Japanese.
func DisplayTitle(titleEnglish, titleJapanese sql.NullString, titleOriginal string) string { func DisplayTitle(titleEnglish, titleJapanese sql.NullString, titleOriginal string) string {
return NullStringOr(titleEnglish, NullStringOr(titleJapanese, titleOriginal)) if titleEnglish.Valid && titleEnglish.String != "" {
return titleEnglish.String
}
if titleOriginal != "" {
return titleOriginal
}
return NullStringOr(titleJapanese, titleOriginal)
} }
func (r GetUserWatchListRow) DisplayTitle() string { func (r GetUserWatchListRow) DisplayTitle() string {

View File

@@ -0,0 +1,30 @@
package db
import (
"database/sql"
"testing"
)
func TestDisplayTitlePrefersOriginalBeforeJapanese(t *testing.T) {
got := DisplayTitle(
sql.NullString{},
sql.NullString{String: "サイバーパンク エッジランナーズ", Valid: true},
"Cyberpunk: Edgerunners",
)
if got != "Cyberpunk: Edgerunners" {
t.Fatalf("DisplayTitle() = %q, want original title", got)
}
}
func TestDisplayTitlePrefersEnglish(t *testing.T) {
got := DisplayTitle(
sql.NullString{String: "Frieren: Beyond Journey's End", Valid: true},
sql.NullString{String: "葬送のフリーレン", Valid: true},
"Sousou no Frieren",
)
if got != "Frieren: Beyond Journey's End" {
t.Fatalf("DisplayTitle() = %q, want English title", got)
}
}

View File

@@ -7,9 +7,6 @@ import (
"fmt" "fmt"
) )
// Note: we intentionally avoid naming this struct SkipSegmentOverride because
// some environments may have an sqlc-generated SkipSegmentOverride model,
// which would cause a redeclare build error.
type SkipSegmentOverrideRow struct { type SkipSegmentOverrideRow struct {
ID string ID string
UserID string UserID string

View File

@@ -1,6 +1,7 @@
package db package db
import ( import (
"context"
"database/sql" "database/sql"
"fmt" "fmt"
@@ -17,7 +18,7 @@ func Open(dbFile string) (*sql.DB, error) {
return nil, fmt.Errorf("failed to open db: %w", err) return nil, fmt.Errorf("failed to open db: %w", err)
} }
// WAL improves concurrency between readers and writers. // WAL improves concurrency between readers and writers.
_, _ = db.Exec("PRAGMA journal_mode=WAL;") _, _ = db.ExecContext(context.Background(), "PRAGMA journal_mode=WAL;")
_, _ = db.Exec("PRAGMA busy_timeout=5000;") _, _ = db.ExecContext(context.Background(), "PRAGMA busy_timeout=5000;")
return db, nil return db, nil
} }

View File

@@ -16,7 +16,7 @@ func TestGetUserWatchlistAnimeIDsFiltersRequestedIDs(t *testing.T) {
} }
defer func() { _ = sqlDB.Close() }() defer func() { _ = sqlDB.Close() }()
_, err = sqlDB.Exec(` _, err = sqlDB.ExecContext(context.Background(), `
CREATE TABLE watch_list_entry ( CREATE TABLE watch_list_entry (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
user_id TEXT NOT NULL, user_id TEXT NOT NULL,

View File

@@ -137,11 +137,6 @@ type AnimeCatalogService interface {
GetTopPicksForYou(ctx context.Context, userID string) (CatalogSectionData, error) GetTopPicksForYou(ctx context.Context, userID string) (CatalogSectionData, error)
} }
type AnimeDiscoverService interface {
GetDiscoverSection(ctx context.Context, userID string, section string) (DiscoverSectionData, error)
GetAiringSchedule(ctx context.Context, userID string) ([]Anime, error)
}
type AnimeSearchService interface { type AnimeSearchService interface {
SearchAdvanced(ctx context.Context, q, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (jikan.SearchResult, error) SearchAdvanced(ctx context.Context, q, animeType, status, orderBy, sort string, genres []int, studioID int, sfw bool, page, limit int) (jikan.SearchResult, error)
GetProducerNameByID(ctx context.Context, id int) (string, error) GetProducerNameByID(ctx context.Context, id int) (string, error)
@@ -153,7 +148,7 @@ type AnimeDetailsService interface {
GetAnimeByID(ctx context.Context, id int) (Anime, error) GetAnimeByID(ctx context.Context, id int) (Anime, error)
GetCharacters(ctx context.Context, id int) ([]CharacterEntry, error) GetCharacters(ctx context.Context, id int) ([]CharacterEntry, error)
GetRecommendations(ctx context.Context, id int) ([]RecommendationEntry, error) GetRecommendations(ctx context.Context, id int) ([]RecommendationEntry, error)
GetRelations(ctx context.Context, id int) ([]jikan.RelationEntry, error) GetRelations(ctx context.Context, id int, mode jikan.WatchOrderMode) ([]jikan.RelationEntry, error)
GetEpisodes(ctx context.Context, id int, page int) (jikan.EpisodesResponse, error) GetEpisodes(ctx context.Context, id int, page int) (jikan.EpisodesResponse, error)
GetAllEpisodes(ctx context.Context, id int) ([]EpisodeData, error) GetAllEpisodes(ctx context.Context, id int) ([]EpisodeData, error)
GetRandomAnime(ctx context.Context) (Anime, error) GetRandomAnime(ctx context.Context) (Anime, error)
@@ -180,17 +175,6 @@ func (d CatalogSectionData) TemplateFragment() string {
return d.Fragment return d.Fragment
} }
type DiscoverSectionData struct {
Animes []Anime
Section string
WatchlistMap map[int64]bool
Fragment string
}
func (d DiscoverSectionData) TemplateFragment() string {
return d.Fragment
}
type AnimeRepository interface { type AnimeRepository interface {
GetUserWatchList(ctx context.Context, userID string) ([]db.GetUserWatchListRow, error) GetUserWatchList(ctx context.Context, userID string) ([]db.GetUserWatchListRow, error)
GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error) GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error)

View File

@@ -49,6 +49,7 @@ type SubtitleItem struct {
type ModeSource struct { type ModeSource struct {
Token string `json:"token"` Token string `json:"token"`
Type string `json:"type,omitempty"`
Subtitles []SubtitleItem `json:"subtitles"` Subtitles []SubtitleItem `json:"subtitles"`
Qualities []string `json:"qualities,omitempty"` Qualities []string `json:"qualities,omitempty"`
} }
@@ -88,6 +89,8 @@ type EpisodeData struct {
type PlaybackRepository interface { type PlaybackRepository interface {
InTx(ctx context.Context, fn func(ctx context.Context, repo PlaybackRepository) error) error InTx(ctx context.Context, fn func(ctx context.Context, repo PlaybackRepository) error) error
UpsertAnime(ctx context.Context, params db.UpsertAnimeParams) (db.Anime, error)
GetAnime(ctx context.Context, id int64) (db.Anime, error)
GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error) GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error)
GetContinueWatchingEntry(ctx context.Context, params db.GetContinueWatchingEntryParams) (db.ContinueWatchingEntry, error) GetContinueWatchingEntry(ctx context.Context, params db.GetContinueWatchingEntryParams) (db.ContinueWatchingEntry, error)
SaveWatchProgress(ctx context.Context, params db.SaveWatchProgressParams) error SaveWatchProgress(ctx context.Context, params db.SaveWatchProgressParams) error

View File

@@ -12,6 +12,7 @@ type StreamSource struct {
type StreamResult struct { type StreamResult struct {
URL string URL string
Referer string Referer string
Type string
Subtitles []Subtitle Subtitles []Subtitle
Qualities []StreamSource Qualities []StreamSource
} }

View File

@@ -2,13 +2,10 @@
package episodes package episodes
import ( import (
"mal/integrations/jikan"
"mal/integrations/playback/allanime" "mal/integrations/playback/allanime"
"mal/internal/config" "mal/internal/config"
"mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
episodeService "mal/internal/episodes/service" episodeService "mal/internal/episodes/service"
"mal/internal/observability"
"go.uber.org/fx" "go.uber.org/fx"
) )
@@ -21,9 +18,7 @@ var Module = fx.Options(
fx.Provide( fx.Provide(
episodeAvailabilityEnabled, episodeAvailabilityEnabled,
fx.Annotate( fx.Annotate(
func(queries *db.Queries, jikanClient *jikan.Client, providers []domain.EpisodeAvailabilityProvider, enabled bool, metrics *observability.Metrics) domain.EpisodeService { episodeService.NewEpisodeService,
return episodeService.NewEpisodeService(queries, jikanClient, providers, enabled, metrics)
},
), ),
), ),
fx.Provide(func(p *allanime.AllAnimeProvider) []domain.EpisodeAvailabilityProvider { fx.Provide(func(p *allanime.AllAnimeProvider) []domain.EpisodeAvailabilityProvider {

View File

@@ -0,0 +1,250 @@
package service
import (
"context"
"database/sql"
"encoding/json"
"time"
"mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *EpisodeService) store(ctx context.Context, anime domain.Anime, jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, source string, now time.Time, providerSuccess bool) (domain.CanonicalEpisodeList, error) {
nextRefreshSQL := nextRefreshAt(anime, now)
episodes := mergeEpisodes(jikanEpisodes, availability, anime.Episodes)
payload := domain.CanonicalEpisodeList{
AnimeID: anime.MalID,
Episodes: episodes,
Source: source,
}
if nextRefreshSQL.Valid {
payload.NextRefreshAt = nextRefreshSQL.Time.Format(time.RFC3339)
}
body, err := json.Marshal(payload)
if err != nil {
return domain.CanonicalEpisodeList{}, err
}
if !s.writeEpisodeAvailabilityCache(ctx, anime, source, body, now, providerSuccess, nextRefreshSQL) {
return payload, nil
}
observability.Info(
"episodes_refresh_success",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
"episodes": len(episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, nil
}
func (s *EpisodeService) writeEpisodeAvailabilityCache(ctx context.Context, anime domain.Anime, source string, body []byte, now time.Time, providerSuccess bool, nextRefreshSQL sql.NullTime) bool {
var retryUntil sql.NullTime
if anime.Airing && providerSuccess {
retryUntil = sql.NullTime{Time: nextRefreshSQL.Time.Add(retryWindow), Valid: nextRefreshSQL.Valid}
}
err := s.queries.UpsertEpisodeAvailabilityCache(ctx, db.UpsertEpisodeAvailabilityCacheParams{
AnimeID: int64(anime.MalID),
Data: string(body),
NextRefreshAt: nextRefreshSQL,
RetryUntilAt: retryUntil,
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastSuccessAt: sql.NullTime{Time: now, Valid: providerSuccess},
FailureCount: 0,
LastError: "",
})
if err == nil {
return true
}
observability.Warn(
"episodes_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
},
err,
)
return false
}
func (s *EpisodeService) markFailure(ctx context.Context, anime domain.Anime, cause error) {
now := s.clock.Now()
next := nextRetryTime(anime, now)
var retryUntil sql.NullTime
nextBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !nextBroadcast.IsZero() {
retryUntil = sql.NullTime{Time: nextBroadcast.Add(retryWindow), Valid: true}
}
var nextSQL sql.NullTime
if !next.IsZero() {
nextSQL = sql.NullTime{Time: next, Valid: true}
}
writeCtx := ctx
if ctx.Err() != nil {
var cancel context.CancelFunc
writeCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
err := s.queries.MarkEpisodeAvailabilityRefreshFailed(writeCtx, db.MarkEpisodeAvailabilityRefreshFailedParams{
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastError: truncate(cause.Error(), 400),
NextRefreshAt: nextSQL,
RetryUntilAt: retryUntil,
AnimeID: int64(anime.MalID),
})
if err != nil {
observability.Warn(
"episodes_mark_failure_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return
}
observability.Warn(
"episodes_refresh_failure_recorded",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_retry": next.Format(time.RFC3339),
},
cause,
)
}
func (s *EpisodeService) getCached(ctx context.Context, animeID int) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(animeID))
if err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": animeID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability", "hit")
return payload, true
}
func (s *EpisodeService) getFreshCached(ctx context.Context, anime domain.Anime) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(anime.MalID))
if err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
return domain.CanonicalEpisodeList{}, false
}
now := s.clock.Now()
if !s.isFreshEpisodeCache(anime, row, now) {
return domain.CanonicalEpisodeList{}, false
}
payload, ok := s.decodeFreshCachedPayload(anime, row.Data)
if !ok {
return domain.CanonicalEpisodeList{}, false
}
if !isCanonicalEpisodePayloadValid(payload, anime.Episodes) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cached_payload_rejected",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"expected_count": anime.Episodes,
"cached_episodes": len(payload.Episodes),
},
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability_fresh", "hit")
observability.Info(
"episodes_cache_served",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"episodes": len(payload.Episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, true
}
func (s *EpisodeService) isFreshEpisodeCache(anime domain.Anime, row db.EpisodeAvailabilityCache, now time.Time) bool {
if row.NextRefreshAt.Valid && !row.NextRefreshAt.Time.After(now) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_due_for_refresh",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_refresh": row.NextRefreshAt.Time.Format(time.RFC3339),
},
)
return false
}
if anime.Airing && row.UpdatedAt.Before(now.Add(-airingFallbackRefreshInterval)) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_too_old_for_airing",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"updated_at": row.UpdatedAt.Format(time.RFC3339),
},
)
return false
}
return true
}
func (s *EpisodeService) decodeFreshCachedPayload(anime domain.Anime, raw string) (domain.CanonicalEpisodeList, bool) {
var payload domain.CanonicalEpisodeList
err := json.Unmarshal([]byte(raw), &payload)
if err == nil {
return payload, true
}
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}

View File

@@ -0,0 +1,133 @@
package service
import (
"fmt"
"sort"
"strconv"
"strings"
"mal/integrations/jikan"
"mal/internal/domain"
)
type episodePartial struct {
title string
filler bool
recap bool
sub bool
dub bool
}
func titleCandidates(anime domain.Anime) []string {
out := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
out = append(out, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
out = append(out, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
out = append(out, syn)
}
}
return out
}
func isCanonicalEpisodePayloadValid(payload domain.CanonicalEpisodeList, expectedCount int) bool {
if expectedCount <= 0 {
return true
}
if len(payload.Episodes) > expectedCount {
return false
}
for _, episode := range payload.Episodes {
if episode.Number <= 0 || episode.Number > expectedCount {
return false
}
}
return true
}
func mergeEpisodes(jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, expectedCount int) []domain.CanonicalEpisode {
byNumber := map[int]episodePartial{}
for i, ep := range jikanEpisodes {
if exceedsExpectedCount(i+1, expectedCount) {
break
}
number, ok := jikanEpisodeNumber(ep, i)
if !ok || exceedsExpectedCount(number, expectedCount) {
continue
}
mergeEpisode(&byNumber, number, func(item *episodePartial) {
item.title = strings.TrimSpace(ep.Title)
item.filler = ep.Filler
item.recap = ep.Recap
})
}
mergeAvailability(&byNumber, availability.Sub, expectedCount, func(item *episodePartial) { item.sub = true })
mergeAvailability(&byNumber, availability.Dub, expectedCount, func(item *episodePartial) { item.dub = true })
numbers := make([]int, 0, len(byNumber))
for number := range byNumber {
numbers = append(numbers, number)
}
sort.Ints(numbers)
episodes := make([]domain.CanonicalEpisode, 0, len(numbers))
for _, number := range numbers {
item := byNumber[number]
title := item.title
if title == "" {
title = fmt.Sprintf("Episode %d", number)
}
episodes = append(episodes, domain.CanonicalEpisode{
Number: number,
Title: title,
HasSub: item.sub,
HasDub: item.dub,
SubOnly: item.sub && !item.dub,
Filler: item.filler,
Recap: item.recap,
})
}
return episodes
}
func mergeEpisode(byNumber *map[int]episodePartial, number int, update func(*episodePartial)) {
item := (*byNumber)[number]
update(&item)
(*byNumber)[number] = item
}
func mergeAvailability(byNumber *map[int]episodePartial, numbers []int, expectedCount int, update func(*episodePartial)) {
for _, number := range numbers {
if number <= 0 || exceedsExpectedCount(number, expectedCount) {
continue
}
mergeEpisode(byNumber, number, update)
}
}
func jikanEpisodeNumber(ep jikan.Episode, index int) (int, bool) {
number, err := strconv.Atoi(strings.TrimSpace(ep.Episode))
if err == nil && number > 0 {
return number, true
}
if index < 0 {
return 0, false
}
return index + 1, true
}
func exceedsExpectedCount(number int, expectedCount int) bool {
return expectedCount > 0 && number > expectedCount
}
func truncate(value string, maxLen int) string {
if len(value) <= maxLen {
return value
}
return value[:maxLen]
}

View File

@@ -0,0 +1,120 @@
package service
import (
"context"
"database/sql"
"errors"
"fmt"
"strings"
"time"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *EpisodeService) providerID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, titles []string) (string, error) {
providerID, found, err := s.cachedProviderID(ctx, anime, provider)
if found || err != nil {
return providerID, err
}
providerID, err = provider.ResolveEpisodeProviderID(ctx, anime.MalID, titles)
if err != nil {
s.cacheProviderIDFailure(ctx, anime, provider, err)
return "", err
}
s.cacheProviderIDSuccess(ctx, anime, provider, providerID)
observability.Info(
"episodes_provider_id_resolved",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": providerID,
},
)
return providerID, nil
}
func (s *EpisodeService) cachedProviderID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider) (string, bool, error) {
row, err := s.queries.GetEpisodeProviderMapping(ctx, db.GetEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
})
if err != nil {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
if errors.Is(err, sql.ErrNoRows) {
return "", false, nil
}
observability.Warn(
"episodes_provider_id_cache_read_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
return "", false, nil
}
if row.FailedUntil.Valid && row.FailedUntil.Time.After(s.clock.Now()) {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
return "", true, fmt.Errorf("cached provider mapping failure active until %s: %s", row.FailedUntil.Time.Format(time.RFC3339), row.LastError)
}
if strings.TrimSpace(row.ProviderShowID) == "" {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
return "", false, nil
}
s.metrics.ObserveCache("episode_provider_mapping", "hit")
observability.Info(
"episodes_provider_id_cache_hit",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": row.ProviderShowID,
},
)
return row.ProviderShowID, true, nil
}
func (s *EpisodeService) cacheProviderIDFailure(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, resolveErr error) {
_ = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: "",
FailedUntil: sql.NullTime{Time: s.clock.Now().Add(time.Hour), Valid: true},
LastError: truncate(resolveErr.Error(), 400),
})
}
func (s *EpisodeService) cacheProviderIDSuccess(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, providerID string) {
err := s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: providerID,
FailedUntil: sql.NullTime{},
LastError: "",
})
if err == nil {
return
}
observability.Warn(
"episodes_provider_id_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
}

View File

@@ -0,0 +1,131 @@
package service
import (
"database/sql"
"strings"
"time"
"mal/internal/domain"
"mal/internal/observability"
)
const (
retryInterval = 15 * time.Minute
retryWindow = 3 * time.Hour
airingFallbackRefreshInterval = 6 * time.Hour
)
func nextRefreshAt(anime domain.Anime, now time.Time) sql.NullTime {
if !anime.Airing {
return sql.NullTime{}
}
// During the hours immediately following a broadcast time, providers can lag.
// Keep retrying for a short window, even if the provider request succeeded.
lastBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !lastBroadcast.IsZero() && now.Before(lastBroadcast.Add(retryWindow)) {
return sql.NullTime{Time: now.Add(retryInterval).UTC(), Valid: true}
}
next := nextBroadcastAfter(anime, now)
if !next.IsZero() {
return sql.NullTime{Time: next, Valid: true}
}
// Broadcast metadata is often missing or wrong for currently airing shows.
// Avoid "never refresh again" caches by falling back to a fixed interval.
return sql.NullTime{Time: now.Add(airingFallbackRefreshInterval).UTC(), Valid: true}
}
func nextRetryTime(anime domain.Anime, now time.Time) time.Time {
broadcast := nextBroadcastBeforeOrAt(anime, now)
if broadcast.IsZero() || now.After(broadcast.Add(retryWindow)) {
return nextBroadcastAfter(anime, now)
}
return now.Add(retryInterval)
}
func nextBroadcastBeforeOrAt(anime domain.Anime, now time.Time) time.Time {
next := nextBroadcastAfter(anime, now.AddDate(0, 0, -7))
if next.IsZero() || next.After(now) {
return time.Time{}
}
return next
}
func nextBroadcastAfter(anime domain.Anime, after time.Time) time.Time {
day := weekdayFromJikan(anime.Broadcast.Day)
if day < 0 || strings.TrimSpace(anime.Broadcast.Time) == "" {
return time.Time{}
}
loc := time.UTC
if tz := strings.TrimSpace(anime.Broadcast.Timezone); tz != "" {
if loaded, err := time.LoadLocation(tz); err == nil {
loc = loaded
} else {
observability.Warn(
"episodes_broadcast_timezone_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"timezone": tz,
},
err,
)
}
}
hour, minute, ok := parseBroadcastTime(anime.Broadcast.Time)
if !ok {
observability.Warn(
"episodes_broadcast_time_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"time": anime.Broadcast.Time,
},
nil,
)
return time.Time{}
}
localAfter := after.In(loc)
daysAhead := (int(day) - int(localAfter.Weekday()) + 7) % 7
candidate := time.Date(localAfter.Year(), localAfter.Month(), localAfter.Day()+daysAhead, hour, minute, 0, 0, loc)
if !candidate.After(localAfter) {
candidate = candidate.AddDate(0, 0, 7)
}
return candidate.UTC()
}
func weekdayFromJikan(day string) time.Weekday {
switch strings.ToLower(strings.TrimSpace(day)) {
case "sundays":
return time.Sunday
case "mondays":
return time.Monday
case "tuesdays":
return time.Tuesday
case "wednesdays":
return time.Wednesday
case "thursdays":
return time.Thursday
case "fridays":
return time.Friday
case "saturdays":
return time.Saturday
default:
return -1
}
}
func parseBroadcastTime(value string) (int, int, bool) {
t, err := time.Parse("15:04", strings.TrimSpace(value))
if err != nil {
return 0, 0, false
}
return t.Hour(), t.Minute(), true
}

View File

@@ -3,24 +3,13 @@ package service
import ( import (
"context" "context"
"database/sql"
"encoding/json"
"errors"
"fmt" "fmt"
"time"
"mal/integrations/jikan" "mal/integrations/jikan"
"mal/internal/db" "mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability" "mal/internal/observability"
"sort"
"strconv"
"strings"
"time"
)
const (
retryInterval = 15 * time.Minute
retryWindow = 3 * time.Hour
airingFallbackRefreshInterval = 6 * time.Hour
) )
type Clock interface { type Clock interface {
@@ -229,337 +218,6 @@ func (s *EpisodeService) fetchProviderAvailability(ctx context.Context, anime do
return domain.EpisodeAvailability{}, "", fmt.Errorf("no episode availability provider matched anime_id=%d", anime.MalID) return domain.EpisodeAvailability{}, "", fmt.Errorf("no episode availability provider matched anime_id=%d", anime.MalID)
} }
func (s *EpisodeService) providerID(ctx context.Context, anime domain.Anime, provider domain.EpisodeAvailabilityProvider, titles []string) (string, error) {
row, err := s.queries.GetEpisodeProviderMapping(ctx, db.GetEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
})
if err == nil {
if row.FailedUntil.Valid && row.FailedUntil.Time.After(s.clock.Now()) {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
return "", fmt.Errorf("cached provider mapping failure active until %s: %s", row.FailedUntil.Time.Format(time.RFC3339), row.LastError)
}
if strings.TrimSpace(row.ProviderShowID) != "" {
s.metrics.ObserveCache("episode_provider_mapping", "hit")
observability.Info(
"episodes_provider_id_cache_hit",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": row.ProviderShowID,
},
)
return row.ProviderShowID, nil
}
s.metrics.ObserveCache("episode_provider_mapping", "miss")
} else if !errors.Is(err, sql.ErrNoRows) {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
observability.Warn(
"episodes_provider_id_cache_read_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
} else {
s.metrics.ObserveCache("episode_provider_mapping", "miss")
}
providerID, err := provider.ResolveEpisodeProviderID(ctx, anime.MalID, titles)
if err != nil {
_ = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: "",
FailedUntil: sql.NullTime{Time: s.clock.Now().Add(time.Hour), Valid: true},
LastError: truncate(err.Error(), 400),
})
return "", err
}
err = s.queries.UpsertEpisodeProviderMapping(ctx, db.UpsertEpisodeProviderMappingParams{
AnimeID: int64(anime.MalID),
Provider: provider.Name(),
ProviderShowID: providerID,
FailedUntil: sql.NullTime{},
LastError: "",
})
if err != nil {
observability.Warn(
"episodes_provider_id_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
},
err,
)
}
observability.Info(
"episodes_provider_id_resolved",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"provider": provider.Name(),
"provider_id": providerID,
},
)
return providerID, nil
}
func (s *EpisodeService) store(ctx context.Context, anime domain.Anime, jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, source string, now time.Time, providerSuccess bool) (domain.CanonicalEpisodeList, error) {
var nextRefreshSQL sql.NullTime
if anime.Airing {
// During the hours immediately following a broadcast time, providers can lag.
// Keep retrying for a short window, even if the provider request succeeded.
lastBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !lastBroadcast.IsZero() && now.Before(lastBroadcast.Add(retryWindow)) {
nextRefreshSQL = sql.NullTime{Time: now.Add(retryInterval).UTC(), Valid: true}
} else {
next := nextBroadcastAfter(anime, now)
if !next.IsZero() {
nextRefreshSQL = sql.NullTime{Time: next, Valid: true}
} else {
// Broadcast metadata is often missing or wrong for currently airing shows.
// Avoid "never refresh again" caches by falling back to a fixed interval.
nextRefreshSQL = sql.NullTime{Time: now.Add(airingFallbackRefreshInterval).UTC(), Valid: true}
}
}
}
episodes := mergeEpisodes(jikanEpisodes, availability, anime.Episodes)
payload := domain.CanonicalEpisodeList{
AnimeID: anime.MalID,
Episodes: episodes,
Source: source,
}
if nextRefreshSQL.Valid {
payload.NextRefreshAt = nextRefreshSQL.Time.Format(time.RFC3339)
}
body, err := json.Marshal(payload)
if err != nil {
return domain.CanonicalEpisodeList{}, err
}
var retryUntil sql.NullTime
if anime.Airing && providerSuccess {
retryUntil = sql.NullTime{Time: nextRefreshSQL.Time.Add(retryWindow), Valid: nextRefreshSQL.Valid}
}
err = s.queries.UpsertEpisodeAvailabilityCache(ctx, db.UpsertEpisodeAvailabilityCacheParams{
AnimeID: int64(anime.MalID),
Data: string(body),
NextRefreshAt: nextRefreshSQL,
RetryUntilAt: retryUntil,
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastSuccessAt: sql.NullTime{Time: now, Valid: providerSuccess},
FailureCount: 0,
LastError: "",
})
if err != nil {
observability.Warn(
"episodes_cache_write_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
},
err,
)
return payload, nil
}
observability.Info(
"episodes_refresh_success",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"source": source,
"episodes": len(episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, nil
}
func (s *EpisodeService) markFailure(ctx context.Context, anime domain.Anime, cause error) {
now := s.clock.Now()
next := nextRetryTime(anime, now)
var retryUntil sql.NullTime
nextBroadcast := nextBroadcastBeforeOrAt(anime, now)
if !nextBroadcast.IsZero() {
retryUntil = sql.NullTime{Time: nextBroadcast.Add(retryWindow), Valid: true}
}
var nextSQL sql.NullTime
if !next.IsZero() {
nextSQL = sql.NullTime{Time: next, Valid: true}
}
writeCtx := ctx
if ctx.Err() != nil {
var cancel context.CancelFunc
writeCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
err := s.queries.MarkEpisodeAvailabilityRefreshFailed(writeCtx, db.MarkEpisodeAvailabilityRefreshFailedParams{
LastAttemptAt: sql.NullTime{Time: now, Valid: true},
LastError: truncate(cause.Error(), 400),
NextRefreshAt: nextSQL,
RetryUntilAt: retryUntil,
AnimeID: int64(anime.MalID),
})
if err != nil {
observability.Warn(
"episodes_mark_failure_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return
}
observability.Warn(
"episodes_refresh_failure_recorded",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_retry": next.Format(time.RFC3339),
},
cause,
)
}
func (s *EpisodeService) getCached(ctx context.Context, animeID int) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(animeID))
if err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": animeID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability", "hit")
return payload, true
}
func (s *EpisodeService) getFreshCached(ctx context.Context, anime domain.Anime) (domain.CanonicalEpisodeList, bool) {
row, err := s.queries.GetEpisodeAvailabilityCache(ctx, int64(anime.MalID))
if err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
return domain.CanonicalEpisodeList{}, false
}
now := s.clock.Now()
if row.NextRefreshAt.Valid && !row.NextRefreshAt.Time.After(now) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_due_for_refresh",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"next_refresh": row.NextRefreshAt.Time.Format(time.RFC3339),
},
)
return domain.CanonicalEpisodeList{}, false
}
if anime.Airing && row.UpdatedAt.Before(now.Add(-airingFallbackRefreshInterval)) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cache_too_old_for_airing",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"updated_at": row.UpdatedAt.Format(time.RFC3339),
},
)
return domain.CanonicalEpisodeList{}, false
}
var payload domain.CanonicalEpisodeList
if err := json.Unmarshal([]byte(row.Data), &payload); err != nil {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Warn(
"episodes_cached_payload_invalid",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
},
err,
)
return domain.CanonicalEpisodeList{}, false
}
if !isCanonicalEpisodePayloadValid(payload, anime.Episodes) {
s.metrics.ObserveCache("episode_availability_fresh", "miss")
observability.Info(
"episodes_cached_payload_rejected",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"expected_count": anime.Episodes,
"cached_episodes": len(payload.Episodes),
},
)
return domain.CanonicalEpisodeList{}, false
}
s.metrics.ObserveCache("episode_availability_fresh", "hit")
observability.Info(
"episodes_cache_served",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"episodes": len(payload.Episodes),
"next_refresh": payload.NextRefreshAt,
},
)
return payload, true
}
func isCanonicalEpisodePayloadValid(payload domain.CanonicalEpisodeList, expectedCount int) bool {
if expectedCount <= 0 {
return true
}
if len(payload.Episodes) > expectedCount {
return false
}
for _, episode := range payload.Episodes {
if episode.Number <= 0 || episode.Number > expectedCount {
return false
}
}
return true
}
func (s *EpisodeService) jikanOnly(ctx context.Context, anime domain.Anime, source string) (domain.CanonicalEpisodeList, error) { func (s *EpisodeService) jikanOnly(ctx context.Context, anime domain.Anime, source string) (domain.CanonicalEpisodeList, error) {
episodes, err := s.jikan.GetAllEpisodes(ctx, anime.MalID) episodes, err := s.jikan.GetAllEpisodes(ctx, anime.MalID)
if err != nil { if err != nil {
@@ -571,201 +229,3 @@ func (s *EpisodeService) jikanOnly(ctx context.Context, anime domain.Anime, sour
Source: source, Source: source,
}, nil }, nil
} }
func titleCandidates(anime domain.Anime) []string {
out := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
out = append(out, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
out = append(out, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
out = append(out, syn)
}
}
return out
}
func mergeEpisodes(jikanEpisodes []jikan.Episode, availability domain.EpisodeAvailability, expectedCount int) []domain.CanonicalEpisode {
type partial struct {
title string
filler bool
recap bool
sub bool
dub bool
}
byNumber := map[int]partial{}
for i, ep := range jikanEpisodes {
if expectedCount > 0 && i >= expectedCount {
break
}
number, ok := jikanEpisodeNumber(ep, i)
if !ok || exceedsExpectedCount(number, expectedCount) {
continue
}
item := byNumber[number]
item.title = strings.TrimSpace(ep.Title)
item.filler = ep.Filler
item.recap = ep.Recap
byNumber[number] = item
}
for _, n := range availability.Sub {
if n <= 0 || exceedsExpectedCount(n, expectedCount) {
continue
}
item := byNumber[n]
item.sub = true
byNumber[n] = item
}
for _, n := range availability.Dub {
if n <= 0 || exceedsExpectedCount(n, expectedCount) {
continue
}
item := byNumber[n]
item.dub = true
byNumber[n] = item
}
numbers := make([]int, 0, len(byNumber))
for number := range byNumber {
numbers = append(numbers, number)
}
sort.Ints(numbers)
episodes := make([]domain.CanonicalEpisode, 0, len(numbers))
for _, number := range numbers {
item := byNumber[number]
title := item.title
if title == "" {
title = fmt.Sprintf("Episode %d", number)
}
episodes = append(episodes, domain.CanonicalEpisode{
Number: number,
Title: title,
HasSub: item.sub,
HasDub: item.dub,
SubOnly: item.sub && !item.dub,
Filler: item.filler,
Recap: item.recap,
})
}
return episodes
}
func jikanEpisodeNumber(ep jikan.Episode, index int) (int, bool) {
number, err := strconv.Atoi(strings.TrimSpace(ep.Episode))
if err == nil && number > 0 {
return number, true
}
if index < 0 {
return 0, false
}
return index + 1, true
}
func exceedsExpectedCount(number int, expectedCount int) bool {
return expectedCount > 0 && number > expectedCount
}
func nextRetryTime(anime domain.Anime, now time.Time) time.Time {
broadcast := nextBroadcastBeforeOrAt(anime, now)
if broadcast.IsZero() || now.After(broadcast.Add(retryWindow)) {
return nextBroadcastAfter(anime, now)
}
return now.Add(retryInterval)
}
func nextBroadcastBeforeOrAt(anime domain.Anime, now time.Time) time.Time {
next := nextBroadcastAfter(anime, now.AddDate(0, 0, -7))
if next.IsZero() || next.After(now) {
return time.Time{}
}
return next
}
func nextBroadcastAfter(anime domain.Anime, after time.Time) time.Time {
day := weekdayFromJikan(anime.Broadcast.Day)
if day < 0 || strings.TrimSpace(anime.Broadcast.Time) == "" {
return time.Time{}
}
loc := time.UTC
if tz := strings.TrimSpace(anime.Broadcast.Timezone); tz != "" {
if loaded, err := time.LoadLocation(tz); err == nil {
loc = loaded
} else {
observability.Warn(
"episodes_broadcast_timezone_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"timezone": tz,
},
err,
)
}
}
hour, minute, ok := parseBroadcastTime(anime.Broadcast.Time)
if !ok {
observability.Warn(
"episodes_broadcast_time_parse_failed",
"episodes",
"",
map[string]any{
"anime_id": anime.MalID,
"time": anime.Broadcast.Time,
},
nil,
)
return time.Time{}
}
localAfter := after.In(loc)
daysAhead := (int(day) - int(localAfter.Weekday()) + 7) % 7
candidate := time.Date(localAfter.Year(), localAfter.Month(), localAfter.Day()+daysAhead, hour, minute, 0, 0, loc)
if !candidate.After(localAfter) {
candidate = candidate.AddDate(0, 0, 7)
}
return candidate.UTC()
}
func weekdayFromJikan(day string) time.Weekday {
switch strings.ToLower(strings.TrimSpace(day)) {
case "sundays":
return time.Sunday
case "mondays":
return time.Monday
case "tuesdays":
return time.Tuesday
case "wednesdays":
return time.Wednesday
case "thursdays":
return time.Thursday
case "fridays":
return time.Friday
case "saturdays":
return time.Saturday
default:
return -1
}
}
func parseBroadcastTime(value string) (int, int, bool) {
t, err := time.Parse("15:04", strings.TrimSpace(value))
if err != nil {
return 0, 0, false
}
return t.Hour(), t.Minute(), true
}
func truncate(value string, maxLen int) string {
if len(value) <= maxLen {
return value
}
return value[:maxLen]
}

View File

@@ -0,0 +1,60 @@
package observability
import (
"go.uber.org/fx/fxevent"
)
type fxLogger struct{}
func NewFxLogger() fxevent.Logger {
return fxLogger{}
}
func (fxLogger) LogEvent(event fxevent.Event) {
eventName, fields, err := describeFXEventError(event)
if err == nil {
return
}
Error(eventName, "fx", "", fields, err)
}
func describeFXEventError(event fxevent.Event) (string, map[string]any, error) {
if ok, eventName, fields, err := describeFXExecutionEventError(event); ok {
return eventName, fields, err
}
return describeFXLifecycleEventError(event)
}
func describeFXExecutionEventError(event fxevent.Event) (bool, string, map[string]any, error) {
switch e := event.(type) {
case *fxevent.Provided:
return true, "fx_provide_failed", map[string]any{"constructor": e.ConstructorName}, e.Err
case *fxevent.Invoked:
return true, "fx_invoke_failed", map[string]any{"function": e.FunctionName}, e.Err
case *fxevent.Run:
return true, "fx_run_failed", map[string]any{"function": e.Name, "kind": e.Kind}, e.Err
case *fxevent.OnStartExecuted:
return true, "fx_on_start_failed", map[string]any{"caller": e.CallerName, "function": e.FunctionName, "runtime": e.Runtime}, e.Err
case *fxevent.OnStopExecuted:
return true, "fx_on_stop_failed", map[string]any{"caller": e.CallerName, "function": e.FunctionName, "runtime": e.Runtime}, e.Err
default:
return false, "", nil, nil
}
}
func describeFXLifecycleEventError(event fxevent.Event) (string, map[string]any, error) {
switch e := event.(type) {
case *fxevent.Started:
return "fx_start_failed", nil, e.Err
case *fxevent.Stopped:
return "fx_stop_failed", nil, e.Err
case *fxevent.RollingBack:
return "fx_rollback_start", nil, e.StartErr
case *fxevent.RolledBack:
return "fx_rollback_failed", nil, e.Err
default:
return "", nil, nil
}
}

View File

@@ -1,5 +1,7 @@
package observability package observability
import "context"
// Small helpers to keep logging consistent and low-friction across the codebase. // Small helpers to keep logging consistent and low-friction across the codebase.
func Info(event string, component string, message string, fields map[string]any) { func Info(event string, component string, message string, fields map[string]any) {
@@ -10,6 +12,14 @@ func Warn(event string, component string, message string, fields map[string]any,
LogJSON(LogLevelWarn, event, component, message, fields, err) LogJSON(LogLevelWarn, event, component, message, fields, err)
} }
func WarnContext(ctx context.Context, event string, component string, message string, fields map[string]any, err error) {
LogContext(ctx, LogLevelWarn, event, component, message, fields, err)
}
func Error(event string, component string, message string, fields map[string]any, err error) { func Error(event string, component string, message string, fields map[string]any, err error) {
LogJSON(LogLevelError, event, component, message, fields, err) LogJSON(LogLevelError, event, component, message, fields, err)
} }
func ErrorContext(ctx context.Context, event string, component string, message string, fields map[string]any, err error) {
LogContext(ctx, LogLevelError, event, component, message, fields, err)
}

View File

@@ -0,0 +1,35 @@
package observability
import (
"errors"
"fmt"
"strings"
"testing"
)
func TestWarnEnrichesSourceAndErrorContext(t *testing.T) {
fields := enrichFields(LogLevelWarn, map[string]any{"anime_id": 123}, wrappedError())
if fields["anime_id"] != 123 {
t.Fatalf("expected existing field to survive, got %#v", fields["anime_id"])
}
source, ok := fields["source"].(string)
if !ok || source == "" {
t.Fatalf("expected source field, got %#v", fields["source"])
}
errorType, ok := fields["error_type"].(string)
if !ok || errorType == "" {
t.Fatalf("expected error_type field, got %#v", fields["error_type"])
}
chain, ok := fields["error_chain"].(string)
if !ok || !strings.Contains(chain, "query anime") || !strings.Contains(chain, "db timeout") {
t.Fatalf("expected wrapped error chain, got %#v", fields["error_chain"])
}
}
func wrappedError() error {
return fmt.Errorf("query anime: %w", errors.New("db timeout"))
}

View File

@@ -2,11 +2,33 @@
package observability package observability
import ( import (
"encoding/json" "context"
"errors"
"fmt"
"log" "log"
"net"
"os"
"path/filepath"
"reflect"
"runtime"
"sort"
"strconv"
"strings"
"time" "time"
) )
const (
ansiReset = "\x1b[0m"
ansiBlue = "\x1b[36m"
ansiStatusBlue = "\x1b[34m"
ansiGreen = "\x1b[32m"
ansiYellow = "\x1b[33m"
ansiOrange = "\x1b[38;5;208m"
ansiRed = "\x1b[31m"
)
var colorLogs = shouldColorLogs()
type LogLevel string type LogLevel string
const ( const (
@@ -25,11 +47,17 @@ type LogEvent struct {
Component string `json:"component,omitempty"` Component string `json:"component,omitempty"`
} }
func init() {
log.SetFlags(0)
}
func LogJSON(level LogLevel, event string, component string, message string, fields map[string]any, err error) { func LogJSON(level LogLevel, event string, component string, message string, fields map[string]any, err error) {
errorValue := "" LogContext(context.TODO(), level, event, component, message, fields, err)
if err != nil { }
errorValue = err.Error()
} func LogContext(ctx context.Context, level LogLevel, event string, component string, message string, fields map[string]any, err error) {
fields = enrichFields(level, fields, err)
fields = enrichRequestFields(ctx, fields)
entry := LogEvent{ entry := LogEvent{
TS: time.Now().UTC().Format(time.RFC3339Nano), TS: time.Now().UTC().Format(time.RFC3339Nano),
@@ -37,23 +65,410 @@ func LogJSON(level LogLevel, event string, component string, message string, fie
Event: event, Event: event,
Message: message, Message: message,
Fields: fields, Fields: fields,
Error: errorValue,
Component: component, Component: component,
} }
// Best-effort. If encoding fails, fall back to a minimal line. if err != nil {
bytes, marshalErr := json.Marshal(entry) entry.Error = err.Error()
if marshalErr != nil { }
// Keep output JSON-only even on failures by constructing a minimal entry.
// Marshal individual strings to ensure proper escaping. log.Print(formatLogEntry(entry))
tsBytes, _ := json.Marshal(time.Now().UTC().Format(time.RFC3339Nano)) }
levelBytes, _ := json.Marshal(level)
eventBytes, _ := json.Marshal("log_marshal_failed") func enrichRequestFields(ctx context.Context, fields map[string]any) map[string]any {
componentBytes, _ := json.Marshal(component) requestContext, ok := RequestContextFromContext(ctx)
errBytes, _ := json.Marshal(marshalErr.Error()) if !ok {
log.Printf(`{"ts":%s,"level":%s,"event":%s,"component":%s,"error":%s}`, tsBytes, levelBytes, eventBytes, componentBytes, errBytes) return fields
}
enriched := cloneFields(fields)
if enriched == nil {
enriched = make(map[string]any, 3)
}
if requestContext.ID != "" {
if _, exists := enriched["request_id"]; !exists {
enriched["request_id"] = requestContext.ID
}
}
if requestContext.Path != "" {
if _, exists := enriched["request_path"]; !exists {
enriched["request_path"] = requestContext.Path
}
}
if requestContext.Route != "" && requestContext.Route != requestContext.Path {
if _, exists := enriched["request_route"]; !exists {
enriched["request_route"] = requestContext.Route
}
}
return enriched
}
func enrichFields(level LogLevel, fields map[string]any, err error) map[string]any {
if level == LogLevelInfo {
return fields
}
enriched := cloneFields(fields)
if enriched == nil {
enriched = make(map[string]any, 3)
}
if _, exists := enriched["source"]; !exists {
if source := callerSource(); source != "" {
enriched["source"] = source
}
}
if err != nil {
if _, exists := enriched["error_type"]; !exists {
if errorType := formatErrorType(err); errorType != "" {
enriched["error_type"] = errorType
}
}
if _, exists := enriched["error_chain"]; !exists {
if chain := formatErrorChain(err); chain != "" {
enriched["error_chain"] = chain
}
}
}
return enriched
}
func callerSource() string {
pcs := make([]uintptr, 8)
n := runtime.Callers(3, pcs)
frames := runtime.CallersFrames(pcs[:n])
for {
frame, more := frames.Next()
if !strings.Contains(frame.File, "/internal/observability/") {
return filepath.Base(frame.File) + ":" + strconv.Itoa(frame.Line)
}
if !more {
return ""
}
}
}
func formatErrorType(err error) string {
errType := reflect.TypeOf(err)
if errType == nil {
return ""
}
return errType.String()
}
func formatErrorChain(err error) string {
parts := make([]string, 0, 4)
for current := err; current != nil; current = errors.Unwrap(current) {
parts = append(parts, current.Error())
if len(parts) == 4 {
break
}
}
if len(parts) <= 1 {
return ""
}
return strings.Join(parts, " -> ")
}
func formatLogEntry(entry LogEvent) string {
if entry.Event == "http_request" {
return formatHTTPRequestLog(entry)
}
parts := []string{entry.TS, formatLogLevel(entry.Level), entry.Event}
if entry.Component != "" {
parts = append(parts, "component="+entry.Component)
}
if entry.Message != "" {
parts = append(parts, quoteIfNeeded(entry.Message))
}
if len(entry.Fields) > 0 {
keys := make([]string, 0, len(entry.Fields))
for key := range entry.Fields {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
parts = append(parts, key+"="+formatFieldValue(entry.Fields[key]))
}
}
if entry.Error != "" {
parts = append(parts, "error="+quoteIfNeeded(entry.Error))
}
return strings.Join(parts, " ")
}
func formatHTTPRequestLog(entry LogEvent) string {
fields := cloneFields(entry.Fields)
status := popField(fields, "status")
method := popField(fields, "method")
path := popField(fields, "path")
duration := popField(fields, "duration_ms")
bytes := popField(fields, "bytes")
route := popField(fields, "route")
query := popField(fields, "query")
clientIP := popField(fields, "client_ip")
parts := []string{entry.TS, formatLogLevel(entry.Level), "http"}
appendNonEmpty(&parts, status)
appendNonEmpty(&parts, strings.TrimSpace(method+" "+path))
appendNonEmpty(&parts, duration)
appendNonEmpty(&parts, bytes)
appendKeyValue(&parts, "route", route)
appendKeyValueQuoted(&parts, "query", query)
appendClientIP(&parts, clientIP)
appendSortedFields(&parts, fields)
if entry.Error != "" {
parts = append(parts, "error="+quoteIfNeeded(entry.Error))
}
return strings.Join(parts, " ")
}
func appendNonEmpty(parts *[]string, value string) {
if value == "" {
return return
} }
log.Print(string(bytes)) *parts = append(*parts, value)
}
func appendKeyValue(parts *[]string, key string, value string) {
if value == "" {
return
}
*parts = append(*parts, key+"="+value)
}
func appendKeyValueQuoted(parts *[]string, key string, value string) {
if value == "" {
return
}
*parts = append(*parts, key+"="+quoteIfNeeded(value))
}
func appendClientIP(parts *[]string, clientIP string) {
if clientIP == "" || isLocalClientIP(clientIP) {
return
}
*parts = append(*parts, "ip="+clientIP)
}
func appendSortedFields(parts *[]string, fields map[string]any) {
if len(fields) == 0 {
return
}
keys := make([]string, 0, len(fields))
for key := range fields {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
*parts = append(*parts, key+"="+formatFieldValue(fields[key]))
}
}
func cloneFields(fields map[string]any) map[string]any {
if len(fields) == 0 {
return nil
}
copyFields := make(map[string]any, len(fields))
for key, value := range fields {
copyFields[key] = value
}
return copyFields
}
func popField(fields map[string]any, key string) string {
if len(fields) == 0 {
return ""
}
value, ok := fields[key]
if !ok {
return ""
}
delete(fields, key)
return formatInlineField(key, value)
}
func formatInlineField(key string, value any) string {
switch key {
case "status":
return formatHTTPStatus(value)
case "duration_ms":
return formatDurationMillis(value)
case "bytes":
return formatBytes(value)
default:
if text, ok := value.(string); ok {
return text
}
return fmt.Sprint(value)
}
}
func formatHTTPStatus(value any) string {
status := fmt.Sprint(value)
if !colorLogs || status == "" {
return status
}
switch status[0] {
case '1':
return ansiStatusBlue + status + ansiReset
case '2':
return ansiGreen + status + ansiReset
case '3':
return ansiYellow + status + ansiReset
case '4':
return ansiOrange + status + ansiReset
case '5':
return ansiRed + status + ansiReset
default:
return status
}
}
func formatDurationMillis(value any) string {
ms, ok := toFloat64(value)
if !ok {
return fmt.Sprint(value)
}
return strconv.FormatFloat(ms, 'f', -1, 64) + "ms"
}
func formatBytes(value any) string {
bytesValue, ok := toFloat64(value)
if !ok {
return fmt.Sprint(value)
}
if bytesValue < 1024 {
return strconv.FormatFloat(bytesValue, 'f', -1, 64) + "B"
}
if bytesValue < 1024*1024 {
return strconv.FormatFloat(bytesValue/1024, 'f', 1, 64) + "KB"
}
return strconv.FormatFloat(bytesValue/(1024*1024), 'f', 1, 64) + "MB"
}
func toFloat64(value any) (float64, bool) {
switch v := value.(type) {
case int:
return float64(v), true
case int32:
return float64(v), true
case int64:
return float64(v), true
case float32:
return float64(v), true
case float64:
return v, true
default:
return 0, false
}
}
func isLocalClientIP(value string) bool {
parsed := net.ParseIP(value)
if parsed == nil {
return false
}
return parsed.IsLoopback()
}
func formatLogLevel(level LogLevel) string {
if colorLogs {
switch level {
case LogLevelWarn:
return ansiYellow + "WARN" + ansiReset
case LogLevelError:
return ansiRed + "ERROR" + ansiReset
default:
return ansiBlue + "INFO" + ansiReset
}
}
switch level {
case LogLevelWarn:
return "WARN"
case LogLevelError:
return "ERROR"
default:
return "INFO"
}
}
func shouldColorLogs() bool {
if strings.TrimSpace(os.Getenv("NO_COLOR")) != "" {
return false
}
if strings.EqualFold(strings.TrimSpace(os.Getenv("TERM")), "dumb") {
return false
}
info, err := os.Stderr.Stat()
if err != nil {
return false
}
return info.Mode()&os.ModeCharDevice != 0
}
func formatFieldValue(value any) string {
switch v := value.(type) {
case string:
return quoteIfNeeded(v)
case time.Duration:
return v.String()
case float32:
return strconv.FormatFloat(float64(v), 'f', -1, 32)
case float64:
return strconv.FormatFloat(v, 'f', -1, 64)
case fmt.Stringer:
return quoteIfNeeded(v.String())
default:
return quoteIfNeeded(fmt.Sprint(value))
}
}
func quoteIfNeeded(value string) string {
if value == "" {
return `""`
}
for _, r := range value {
if r == '=' || r == ' ' || r == '\t' || r == '\n' || r == '"' {
return strconv.Quote(value)
}
}
return value
} }

View File

@@ -0,0 +1,60 @@
package observability
import (
"strings"
"testing"
)
func TestFormatLogEntryFormatsHTTPRequestCompactly(t *testing.T) {
line := formatLogEntry(LogEvent{
TS: "2026-06-11T12:57:39.557972Z",
Level: LogLevelInfo,
Event: "http_request",
Fields: map[string]any{
"bytes": 56198,
"client_ip": "127.0.0.1",
"duration_ms": 9.419,
"method": "GET",
"path": "/api/catalog/top-pick",
"status": 200,
},
})
checks := []string{
"2026-06-11T12:57:39.557972Z INFO http 200 GET /api/catalog/top-pick 9.419ms 54.9KB",
}
for _, check := range checks {
if !strings.Contains(line, check) {
t.Fatalf("line %q missing %q", line, check)
}
}
if strings.Contains(line, "client_ip=") {
t.Fatalf("line should omit loopback ip: %q", line)
}
}
func TestFormatHTTPStatusColorsByStatusFamily(t *testing.T) {
previousColorLogs := colorLogs
colorLogs = true
t.Cleanup(func() {
colorLogs = previousColorLogs
})
tests := map[any]string{
101: ansiStatusBlue + "101" + ansiReset,
200: ansiGreen + "200" + ansiReset,
302: ansiYellow + "302" + ansiReset,
404: ansiOrange + "404" + ansiReset,
500: ansiRed + "500" + ansiReset,
"unknown": "unknown",
}
for input, want := range tests {
got := formatHTTPStatus(input)
if got != want {
t.Fatalf("formatHTTPStatus(%v) = %q, want %q", input, got, want)
}
}
}

View File

@@ -1,6 +1,7 @@
package observability package observability
import ( import (
"context"
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -17,7 +18,7 @@ func TestMetricsHandlerRendersPrometheusFamilies(t *testing.T) {
metrics.ObserveCache("jikan", "hit") metrics.ObserveCache("jikan", "hit")
metrics.ObserveCache("episode_availability", "miss") metrics.ObserveCache("episode_availability", "miss")
req := httptest.NewRequest(http.MethodGet, "/metrics", nil) req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/metrics", nil)
rec := httptest.NewRecorder() rec := httptest.NewRecorder()
metrics.Handler().ServeHTTP(rec, req) metrics.Handler().ServeHTTP(rec, req)

View File

@@ -0,0 +1,32 @@
package observability
import "context"
type requestContextKey struct{}
type RequestContext struct {
ID string
Path string
Route string
}
func WithRequestContext(ctx context.Context, requestID string, path string, route string) context.Context {
if ctx == nil {
return nil
}
return context.WithValue(ctx, requestContextKey{}, RequestContext{
ID: requestID,
Path: path,
Route: route,
})
}
func RequestContextFromContext(ctx context.Context) (RequestContext, bool) {
if ctx == nil {
return RequestContext{}, false
}
requestContext, ok := ctx.Value(requestContextKey{}).(RequestContext)
return requestContext, ok
}

View File

@@ -3,9 +3,11 @@ package handler
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"io" "io"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability"
"mal/internal/server" "mal/internal/server"
netutil "mal/pkg/net" netutil "mal/pkg/net"
"net/http" "net/http"
@@ -37,7 +39,6 @@ func NewPlaybackHandler(svc domain.PlaybackService, animeSvc domain.AnimePlaybac
} }
func (h *PlaybackHandler) Register(r *gin.Engine) { func (h *PlaybackHandler) Register(r *gin.Engine) {
r.GET("/anime/:id/watch", h.HandleWatchPage) r.GET("/anime/:id/watch", h.HandleWatchPage)
r.POST("/api/watch-progress", h.HandleSaveProgress) r.POST("/api/watch-progress", h.HandleSaveProgress)
r.POST("/api/watch-complete", h.HandleWatchComplete) r.POST("/api/watch-complete", h.HandleWatchComplete)
@@ -302,6 +303,10 @@ func (h *PlaybackHandler) HandleProxyStream(c *gin.Context) {
resp, err := h.streamingClient.Do(req) resp, err := h.streamingClient.Do(req)
if err != nil { if err != nil {
if !errors.Is(err, context.Canceled) {
observability.ErrorContext(c.Request.Context(), "proxy_stream_upstream_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
}
c.Status(http.StatusBadGateway) c.Status(http.StatusBadGateway)
return return
} }
@@ -310,11 +315,15 @@ func (h *PlaybackHandler) HandleProxyStream(c *gin.Context) {
if isHLSPlaylistResponse(targetURL, resp.Header) { if isHLSPlaylistResponse(targetURL, resp.Header) {
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2)) body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil { if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_stream_playlist_read_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway) c.Status(http.StatusBadGateway)
return return
} }
rewritten, err := h.rewriteHLSPlaylist(string(body), targetURL, referer) rewritten, err := h.rewriteHLSPlaylist(string(body), targetURL, referer)
if err != nil { if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_stream_playlist_rewrite_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway) c.Status(http.StatusBadGateway)
return return
} }
@@ -415,7 +424,9 @@ func (h *PlaybackHandler) proxyPlaylistURI(rawURI string, baseURL *url.URL, refe
if err != nil { if err != nil {
return "", err return "", err
} }
return "/watch/proxy/stream?token=" + url.QueryEscape(token), nil params := url.Values{}
params.Set("token", token)
return "/watch/proxy/stream?" + params.Encode(), nil
} }
func copyProxyHeaders(dst http.Header, src http.Header) { func copyProxyHeaders(dst http.Header, src http.Header) {
@@ -482,6 +493,10 @@ func (h *PlaybackHandler) HandleProxySubtitle(c *gin.Context) {
resp, err := h.proxyClient.Do(req) resp, err := h.proxyClient.Do(req)
if err != nil { if err != nil {
if !errors.Is(err, context.Canceled) {
observability.ErrorContext(c.Request.Context(), "proxy_subtitle_upstream_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
}
c.Status(http.StatusBadGateway) c.Status(http.StatusBadGateway)
return return
} }
@@ -489,6 +504,8 @@ func (h *PlaybackHandler) HandleProxySubtitle(c *gin.Context) {
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2)) body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.MiB2))
if err != nil { if err != nil {
observability.ErrorContext(c.Request.Context(), "proxy_subtitle_read_failed", "playback", "", map[string]any{"target_url": targetURL}, err)
_ = c.Error(err).SetType(gin.ErrorTypePrivate)
c.Status(http.StatusBadGateway) c.Status(http.StatusBadGateway)
return return
} }

View File

@@ -1,7 +1,6 @@
package playback package playback
import ( import (
"mal/integrations/jikan"
"mal/integrations/playback/allanime" "mal/integrations/playback/allanime"
"mal/internal/config" "mal/internal/config"
"mal/internal/domain" "mal/internal/domain"
@@ -18,14 +17,8 @@ func provideProxyTokenKey(cfg config.Config) ProxyTokenKey {
var Module = fx.Options( var Module = fx.Options(
fx.Provide( fx.Provide(
NewPlaybackRepository, NewPlaybackRepository,
fx.Annotate( NewPlaybackService,
func(repo domain.PlaybackRepository, providers []domain.Provider, jikan *jikan.Client, episodeSvc domain.EpisodeService, auditSvc domain.AuditService, proxyTokenKey ProxyTokenKey) domain.PlaybackService { handler.NewPlaybackHandler,
return NewPlaybackService(repo, providers, jikan, episodeSvc, auditSvc, proxyTokenKey)
},
),
func(svc domain.PlaybackService, animeSvc domain.AnimePlaybackService) *handler.PlaybackHandler {
return handler.NewPlaybackHandler(svc, animeSvc)
},
), ),
fx.Provide( fx.Provide(
server.AsRouteRegister(func(h *handler.PlaybackHandler) server.RouteRegister { server.AsRouteRegister(func(h *handler.PlaybackHandler) server.RouteRegister {

View File

@@ -0,0 +1,193 @@
package playback
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"strconv"
"github.com/google/uuid"
"mal/internal/db"
"mal/internal/domain"
"mal/internal/observability"
)
func (s *playbackService) loadWatchProgress(ctx context.Context, userID string, animeID int, totalEpisodes int, episode string) (float64, string, []int64) {
if userID == "" {
return 0, "", nil
}
entry, err := s.repo.GetWatchListEntry(ctx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
watchlistStatus := ""
var watchlistIDs []int64
startTime := 0.0
if err == nil {
watchlistStatus = entry.Status
watchlistIDs = []int64{entry.AnimeID}
if resumeTimeForEpisode(entry.CurrentEpisode, entry.CurrentTimeSeconds, totalEpisodes, episode) > 0 {
startTime = entry.CurrentTimeSeconds
}
}
if startTime > 0 {
return startTime, watchlistStatus, watchlistIDs
}
cwEntry, err := s.repo.GetContinueWatchingEntry(ctx, db.GetContinueWatchingEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
startTime = resumeTimeForEpisode(cwEntry.CurrentEpisode, cwEntry.CurrentTimeSeconds, totalEpisodes, episode)
}
return startTime, watchlistStatus, watchlistIDs
}
func resumeTimeForEpisode(currentEpisode sql.NullInt64, currentTimeSeconds float64, totalEpisodes int, requestedEpisode string) float64 {
if !currentEpisode.Valid {
return 0
}
if strconv.FormatInt(currentEpisode.Int64, 10) == requestedEpisode {
return currentTimeSeconds
}
if totalEpisodes > 0 && requestedEpisode == strconv.Itoa(totalEpisodes) && currentEpisode.Int64 == int64(totalEpisodes) {
return currentTimeSeconds
}
return 0
}
func (s *playbackService) CompleteAnime(ctx context.Context, userID string, animeID int64) error {
if err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
entry, err := repo.GetWatchListEntry(txCtx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: animeID,
})
if err != nil || entry.Status != "completed" {
_, err = repo.UpsertWatchListEntry(txCtx, db.UpsertWatchListEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Status: "completed",
CurrentEpisode: entry.CurrentEpisode,
CurrentTimeSeconds: entry.CurrentTimeSeconds,
})
if err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
if err := s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_completed",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
}); err != nil {
observability.Warn(
"audit_record_failed",
"playback",
"",
map[string]any{"user_id": userID, "anime_id": animeID, "action": "watch_completed"},
err,
)
}
return nil
}
func (s *playbackService) SaveProgress(ctx context.Context, userID string, animeID int64, episode int, timeSeconds float64) error {
err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
if _, err := repo.GetAnime(txCtx, animeID); err != nil {
if _, err := repo.UpsertAnime(txCtx, minimalAnimeParams(animeID)); err != nil {
return err
}
}
_, err := repo.UpsertContinueWatchingEntry(txCtx, db.UpsertContinueWatchingEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
CurrentEpisode: sql.NullInt64{Int64: int64(episode), Valid: true},
CurrentTimeSeconds: timeSeconds,
DurationSeconds: sql.NullFloat64{Valid: false},
})
return err
})
if err != nil {
return err
}
metadataBytes, marshalErr := json.Marshal(struct {
Episode int `json:"episode"`
TimeSeconds float64 `json:"time_seconds"`
}{Episode: episode, TimeSeconds: timeSeconds})
if marshalErr == nil {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
MetadataJSON: metadataBytes,
})
} else {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
})
}
observability.Info("watch_progress_saved", "playback", "", map[string]any{
"anime_id": animeID,
"episode": episode,
"time_seconds": timeSeconds,
"user_id": userID,
})
return nil
}
func (s *playbackService) ensureAnimeRow(ctx context.Context, anime domain.Anime) {
if _, err := s.repo.GetAnime(ctx, int64(anime.MalID)); err == nil {
return
}
_, _ = s.repo.UpsertAnime(ctx, animeParams(anime))
}
func animeParams(anime domain.Anime) db.UpsertAnimeParams {
durationSeconds := anime.DurationSeconds()
duration := sql.NullFloat64{Valid: durationSeconds > 0}
if duration.Valid {
duration.Float64 = durationSeconds
}
return db.UpsertAnimeParams{
ID: int64(anime.MalID),
TitleOriginal: anime.Title,
TitleEnglish: sql.NullString{String: anime.TitleEnglish, Valid: anime.TitleEnglish != ""},
TitleJapanese: sql.NullString{String: anime.TitleJapanese, Valid: anime.TitleJapanese != ""},
ImageUrl: anime.ImageURL(),
Airing: sql.NullBool{Bool: anime.Airing, Valid: true},
DurationSeconds: duration,
}
}
func minimalAnimeParams(animeID int64) db.UpsertAnimeParams {
return db.UpsertAnimeParams{
ID: animeID,
TitleOriginal: fmt.Sprintf("Anime %d", animeID),
Airing: sql.NullBool{Valid: false},
}
}

View File

@@ -0,0 +1,69 @@
package playback
import (
"crypto/rand"
"encoding/base64"
"fmt"
"sync"
"time"
)
type proxyTokenTarget struct {
targetURL string
referer string
scope string
expiresAt time.Time
}
type proxyTokenStore struct {
mu sync.Mutex
tokens map[string]proxyTokenTarget
}
func newProxyTokenStore() *proxyTokenStore {
return &proxyTokenStore{
tokens: make(map[string]proxyTokenTarget),
}
}
func (s *proxyTokenStore) create(targetURL, referer, scope string, ttl time.Duration, now time.Time) (string, error) {
tokenBytes := make([]byte, 32)
if _, err := rand.Read(tokenBytes); err != nil {
return "", fmt.Errorf("generate proxy token: %w", err)
}
token := base64.RawURLEncoding.EncodeToString(tokenBytes)
s.mu.Lock()
defer s.mu.Unlock()
s.pruneExpiredLocked(now)
s.tokens[token] = proxyTokenTarget{
targetURL: targetURL,
referer: referer,
scope: scope,
expiresAt: now.Add(ttl),
}
return token, nil
}
func (s *proxyTokenStore) resolve(token string, now time.Time) (proxyTokenTarget, error) {
s.mu.Lock()
defer s.mu.Unlock()
target, ok := s.tokens[token]
if !ok {
return proxyTokenTarget{}, fmt.Errorf("invalid proxy token")
}
if !target.expiresAt.After(now) {
delete(s.tokens, token)
return proxyTokenTarget{}, fmt.Errorf("proxy token expired")
}
return target, nil
}
func (s *proxyTokenStore) pruneExpiredLocked(now time.Time) {
for token, target := range s.tokens {
if !target.expiresAt.After(now) {
delete(s.tokens, token)
}
}
}

View File

@@ -23,6 +23,14 @@ func (r *playbackRepository) InTx(ctx context.Context, fn func(ctx context.Conte
}, fn) }, fn)
} }
func (r *playbackRepository) UpsertAnime(ctx context.Context, params db.UpsertAnimeParams) (db.Anime, error) {
return r.queries.UpsertAnime(ctx, params)
}
func (r *playbackRepository) GetAnime(ctx context.Context, id int64) (db.Anime, error) {
return r.queries.GetAnime(ctx, id)
}
func (r *playbackRepository) GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error) { func (r *playbackRepository) GetWatchListEntry(ctx context.Context, params db.GetWatchListEntryParams) (db.WatchListEntry, error) {
return r.queries.GetWatchListEntry(ctx, params) return r.queries.GetWatchListEntry(ctx, params)
} }

View File

@@ -3,26 +3,12 @@ package playback
import ( import (
"context" "context"
"crypto/rand"
"database/sql"
"encoding/base64"
"encoding/json"
"fmt" "fmt"
"io"
"mal/integrations/jikan" "mal/integrations/jikan"
"mal/internal/db"
"mal/internal/domain" "mal/internal/domain"
"mal/internal/observability"
netutil "mal/pkg/net" netutil "mal/pkg/net"
"net/http" "net/http"
"net/url"
"sort"
"strconv"
"strings"
"sync"
"time" "time"
"github.com/google/uuid"
) )
type playbackService struct { type playbackService struct {
@@ -38,66 +24,6 @@ type playbackService struct {
type ProxyTokenKey string type ProxyTokenKey string
type proxyTokenTarget struct {
targetURL string
referer string
scope string
expiresAt time.Time
}
type proxyTokenStore struct {
mu sync.Mutex
tokens map[string]proxyTokenTarget
}
func newProxyTokenStore() *proxyTokenStore {
return &proxyTokenStore{
tokens: make(map[string]proxyTokenTarget),
}
}
func (s *proxyTokenStore) create(targetURL, referer, scope string, ttl time.Duration, now time.Time) (string, error) {
tokenBytes := make([]byte, 32)
if _, err := rand.Read(tokenBytes); err != nil {
return "", fmt.Errorf("generate proxy token: %w", err)
}
token := base64.RawURLEncoding.EncodeToString(tokenBytes)
s.mu.Lock()
defer s.mu.Unlock()
s.pruneExpiredLocked(now)
s.tokens[token] = proxyTokenTarget{
targetURL: targetURL,
referer: referer,
scope: scope,
expiresAt: now.Add(ttl),
}
return token, nil
}
func (s *proxyTokenStore) resolve(token string, now time.Time) (proxyTokenTarget, error) {
s.mu.Lock()
defer s.mu.Unlock()
target, ok := s.tokens[token]
if !ok {
return proxyTokenTarget{}, fmt.Errorf("invalid proxy token")
}
if !target.expiresAt.After(now) {
delete(s.tokens, token)
return proxyTokenTarget{}, fmt.Errorf("proxy token expired")
}
return target, nil
}
func (s *proxyTokenStore) pruneExpiredLocked(now time.Time) {
for token, target := range s.tokens {
if !target.expiresAt.After(now) {
delete(s.tokens, token)
}
}
}
func NewPlaybackService(repo domain.PlaybackRepository, providers []domain.Provider, jikan *jikan.Client, episodes domain.EpisodeService, auditSvc domain.AuditService, proxyTokenKey ProxyTokenKey) domain.PlaybackService { func NewPlaybackService(repo domain.PlaybackRepository, providers []domain.Provider, jikan *jikan.Client, episodes domain.EpisodeService, auditSvc domain.AuditService, proxyTokenKey ProxyTokenKey) domain.PlaybackService {
return &playbackService{ return &playbackService{
repo: repo, repo: repo,
@@ -132,408 +58,11 @@ func (s *playbackService) ResolveProxyToken(token string, scope string) (string,
return target.targetURL, target.referer, nil return target.targetURL, target.referer, nil
} }
func (s *playbackService) BuildWatchData(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string, userID string) (domain.WatchPageData, error) {
// 1. Get Anime details for total episodes and titles
anime, err := s.jikan.GetAnimeByID(ctx, animeID)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch anime: %w", err)
}
// 2. Resolve streams from providers
searchTitles := []string{anime.Title}
if anime.TitleEnglish != "" && anime.TitleEnglish != anime.Title {
searchTitles = append(searchTitles, anime.TitleEnglish)
}
if anime.TitleJapanese != "" {
searchTitles = append(searchTitles, anime.TitleJapanese)
}
for _, syn := range anime.TitleSynonyms {
if syn != "" && syn != anime.Title && syn != anime.TitleEnglish && syn != anime.TitleJapanese {
searchTitles = append(searchTitles, syn)
}
}
canonicalEpisodes, err := s.episodes.GetCanonicalEpisodes(ctx, domain.Anime{Anime: anime}, false)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch episodes: %w", err)
}
requestedMode := mode
modeSwitchedFrom := ""
if epNum, parseErr := strconv.Atoi(episode); parseErr == nil && requestedMode == "dub" {
for _, ep := range canonicalEpisodes.Episodes {
if ep.Number == epNum && !ep.HasDub && ep.HasSub {
mode = "sub"
modeSwitchedFrom = requestedMode
break
}
}
}
modeSources := map[string]domain.ModeSource{}
var result *domain.StreamResult
for _, m := range []string{"sub", "dub"} {
for _, p := range s.providers {
res, err := p.GetStreams(ctx, animeID, searchTitles, episode, m)
if err != nil || res == nil {
continue
}
var subItems []domain.SubtitleItem
for _, sub := range res.Subtitles {
subToken, _ := s.SignProxyToken(sub.URL, res.Referer, "subtitle")
subItems = append(subItems, domain.SubtitleItem{
Lang: sub.Label,
Token: subToken,
})
}
streamToken, _ := s.SignProxyToken(res.URL, res.Referer, "stream")
modeSources[m] = domain.ModeSource{
Token: streamToken,
Subtitles: subItems,
}
if m == mode {
result = res
}
break
}
}
if len(modeSources) == 0 {
return domain.WatchPageData{}, fmt.Errorf("no streams found")
}
if result == nil {
return domain.WatchPageData{}, fmt.Errorf("no streams found for mode %s", mode)
}
// 3. Get start time from progress
startTime := 0.0
var watchlistStatus string
var watchlistIDs []int64
if userID != "" {
entry, err := s.repo.GetWatchListEntry(ctx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
watchlistStatus = entry.Status
watchlistIDs = []int64{entry.AnimeID}
if entry.CurrentEpisode.Valid && strconv.FormatInt(entry.CurrentEpisode.Int64, 10) == episode {
startTime = entry.CurrentTimeSeconds
} else if anime.Episodes > 0 && episode == strconv.Itoa(anime.Episodes) && entry.CurrentEpisode.Valid && entry.CurrentEpisode.Int64 == int64(anime.Episodes) {
startTime = entry.CurrentTimeSeconds
}
}
// Fall back to continue_watching_entry for progress if not in watchlist
if startTime == 0 {
cwEntry, err := s.repo.GetContinueWatchingEntry(ctx, db.GetContinueWatchingEntryParams{
UserID: userID,
AnimeID: int64(animeID),
})
if err == nil {
if cwEntry.CurrentEpisode.Valid && strconv.FormatInt(cwEntry.CurrentEpisode.Int64, 10) == episode {
startTime = cwEntry.CurrentTimeSeconds
} else if anime.Episodes > 0 && episode == strconv.Itoa(anime.Episodes) && cwEntry.CurrentEpisode.Valid && cwEntry.CurrentEpisode.Int64 == int64(anime.Episodes) {
startTime = cwEntry.CurrentTimeSeconds
}
}
}
}
// 5. Build provider data
streams := []domain.ProviderStream{
{
Name: "Primary",
Quality: "Auto",
MalID: animeID,
IsCurrent: true,
},
}
go s.warmStreamURL(result.URL, result.Referer)
// 6. Resolve relations/seasons
relations, _ := s.jikan.GetFullRelations(ctx, animeID)
var seasons []domain.SeasonEntry
tvCounter := 1
for _, rel := range relations {
if strings.ToLower(rel.Anime.Type) == "tv" || strings.ToLower(rel.Anime.Type) == "movie" {
seasons = append(seasons, domain.SeasonEntry{
MalID: rel.Anime.MalID,
Title: rel.Anime.DisplayTitle(),
Prefix: rel.Relation,
IsCurrent: rel.IsCurrent,
})
if rel.Relation == "TV" {
seasons[len(seasons)-1].Prefix = fmt.Sprintf("S%d", tvCounter)
tvCounter++
}
}
}
// Final assembly
segments := s.fetchSkipSegments(ctx, userID, animeID, episode)
watchData := domain.WatchData{
MalID: animeID,
Title: anime.DisplayTitle(),
CurrentEpisode: episode,
StartTimeSeconds: startTime,
Episodes: canonicalEpisodes.Episodes,
Providers: []domain.ProviderData{
{Streams: streams},
},
ModeSources: modeSources,
InitialMode: mode,
ModeSwitchedFrom: modeSwitchedFrom,
AvailableModes: func() []string {
var modes []string
for m := range modeSources {
modes = append(modes, m)
}
sort.Strings(modes)
return modes
}(),
Segments: segments,
Airing: anime.Airing,
}
return domain.WatchPageData{
WatchData: watchData,
Anime: domain.Anime{Anime: anime},
Episodes: canonicalEpisodes.Episodes,
CurrentEpID: episode,
WatchlistStatus: watchlistStatus,
WatchlistIDs: watchlistIDs,
Seasons: seasons,
}, nil
}
func (s *playbackService) CompleteAnime(ctx context.Context, userID string, animeID int64) error {
if err := s.repo.InTx(ctx, func(txCtx context.Context, repo domain.PlaybackRepository) error {
entry, err := repo.GetWatchListEntry(txCtx, db.GetWatchListEntryParams{
UserID: userID,
AnimeID: animeID,
})
if err != nil || entry.Status != "completed" {
_, err = repo.UpsertWatchListEntry(txCtx, db.UpsertWatchListEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Status: "completed",
CurrentEpisode: entry.CurrentEpisode,
CurrentTimeSeconds: entry.CurrentTimeSeconds,
})
if err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
if err := s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_completed",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
}); err != nil {
observability.Warn(
"audit_record_failed",
"playback",
"",
map[string]any{"user_id": userID, "anime_id": animeID, "action": "watch_completed"},
err,
)
}
return nil
}
func (s *playbackService) SaveProgress(ctx context.Context, userID string, animeID int64, episode int, timeSeconds float64) error {
_, err := s.repo.UpsertContinueWatchingEntry(ctx, db.UpsertContinueWatchingEntryParams{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
CurrentEpisode: sql.NullInt64{Int64: int64(episode), Valid: true},
CurrentTimeSeconds: timeSeconds,
DurationSeconds: sql.NullFloat64{Valid: false},
})
if err != nil {
return err
}
metadataBytes, marshalErr := json.Marshal(struct {
Episode int `json:"episode"`
TimeSeconds float64 `json:"time_seconds"`
}{Episode: episode, TimeSeconds: timeSeconds})
if marshalErr == nil {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
MetadataJSON: metadataBytes,
})
} else {
_ = s.auditSvc.Record(ctx, domain.AuditEvent{
UserID: userID,
Action: "watch_progress_saved",
ResourceType: "anime",
ResourceID: strconv.FormatInt(animeID, 10),
})
}
observability.Info("watch_progress_saved", "playback", "", map[string]any{
"anime_id": animeID,
"episode": episode,
"time_seconds": timeSeconds,
"user_id": userID,
})
return nil
}
func (s *playbackService) UpsertSkipSegmentOverride(ctx context.Context, userID string, animeID int64, episode int, skipType string, startTime, endTime float64) error {
if userID == "" {
return fmt.Errorf("not authenticated")
}
if animeID <= 0 || episode <= 0 {
return fmt.Errorf("invalid anime/episode")
}
t := strings.ToLower(strings.TrimSpace(skipType))
switch t {
case "op", "opening", "intro":
t = "op"
case "ed", "ending", "outro":
t = "ed"
default:
return fmt.Errorf("invalid skip_type")
}
if !(startTime >= 0) || !(endTime > startTime) {
return fmt.Errorf("invalid interval")
}
// let the player-side filters ignore obviously wrong durations, but keep some sanity.
if endTime-startTime < 5 || endTime-startTime > 10*60 {
return fmt.Errorf("interval duration out of range")
}
return s.repo.UpsertSkipSegmentOverride(ctx, db.SkipSegmentOverrideRow{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Episode: int64(episode),
SkipType: t,
StartTime: startTime,
EndTime: endTime,
})
}
func (s *playbackService) fetchSkipSegments(ctx context.Context, userID string, malID int, episode string) []domain.SkipSegment {
if malID <= 0 || strings.TrimSpace(episode) == "" {
return []domain.SkipSegment{}
}
segments := []domain.SkipSegment{}
endpoint := fmt.Sprintf("https://api.aniskip.com/v1/skip-times/%s/%s?types=op&types=ed", url.PathEscape(strconv.Itoa(malID)), url.PathEscape(episode))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err == nil {
req.Header.Set("User-Agent", netutil.Generic)
if resp, err := s.httpClient.Do(req); err == nil {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode == http.StatusOK {
if body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.KiB512)); err == nil {
type resultItem struct {
SkipType string `json:"skip_type"`
Interval struct {
StartTime float64 `json:"start_time"`
EndTime float64 `json:"end_time"`
} `json:"interval"`
}
type apiResponse struct {
Found bool `json:"found"`
Result []resultItem `json:"results"`
}
var parsed apiResponse
if err := json.Unmarshal(body, &parsed); err == nil && parsed.Found && len(parsed.Result) > 0 {
segments = make([]domain.SkipSegment, 0, len(parsed.Result))
for _, r := range parsed.Result {
skipType := strings.ToLower(r.SkipType)
switch skipType {
case "op":
skipType = "opening"
case "ed":
skipType = "ending"
}
segments = append(segments, domain.SkipSegment{
Type: skipType,
Start: r.Interval.StartTime,
End: r.Interval.EndTime,
Source: "aniskip",
})
}
}
}
}
}
}
epNum, _ := strconv.ParseInt(strings.TrimSpace(episode), 10, 64)
if userID != "" && epNum > 0 {
if ok, err := s.repo.HasSkipSegmentOverrideTable(ctx); err == nil && ok {
if overrides, err := s.repo.ListSkipSegmentOverrides(ctx, userID, int64(malID), epNum); err == nil {
// Build map keyed by normalized type ("opening"/"ending")
overrideByType := make(map[string]domain.SkipSegment, len(overrides))
for _, o := range overrides {
t := strings.ToLower(strings.TrimSpace(o.SkipType))
switch t {
case "op", "opening", "intro":
t = "opening"
case "ed", "ending", "outro":
t = "ending"
default:
continue
}
overrideByType[t] = domain.SkipSegment{
Type: t,
Start: o.StartTime,
End: o.EndTime,
Source: "override",
}
}
if len(overrideByType) > 0 {
merged := make([]domain.SkipSegment, 0, len(segments)+len(overrideByType))
seen := map[string]bool{}
for _, seg := range segments {
if o, ok := overrideByType[seg.Type]; ok {
merged = append(merged, o)
seen[seg.Type] = true
} else {
merged = append(merged, seg)
seen[seg.Type] = true
}
}
for t, o := range overrideByType {
if !seen[t] {
merged = append(merged, o)
}
}
segments = merged
}
}
}
}
return segments
}
func (s *playbackService) warmStreamURL(targetURL, referer string) { func (s *playbackService) warmStreamURL(targetURL, referer string) {
req, err := http.NewRequest(http.MethodGet, targetURL, nil) ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, targetURL, nil)
if err != nil { if err != nil {
return return
} }
@@ -542,10 +71,6 @@ func (s *playbackService) warmStreamURL(targetURL, referer string) {
} }
req.Header.Set("User-Agent", netutil.Firefox121) req.Header.Set("User-Agent", netutil.Firefox121)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req = req.WithContext(ctx)
resp, err := s.httpClient.Do(req) resp, err := s.httpClient.Do(req)
if err != nil { if err != nil {
return return

View File

@@ -0,0 +1,210 @@
package playback
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"github.com/google/uuid"
"mal/internal/db"
"mal/internal/domain"
netutil "mal/pkg/net"
)
func normalizeSkipType(skipType string) (string, error) {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op", "opening", "intro":
return "op", nil
case "ed", "ending", "outro":
return "ed", nil
default:
return "", fmt.Errorf("invalid skip_type")
}
}
func (s *playbackService) UpsertSkipSegmentOverride(ctx context.Context, userID string, animeID int64, episode int, skipType string, startTime, endTime float64) error {
if userID == "" {
return fmt.Errorf("not authenticated")
}
if animeID <= 0 || episode <= 0 {
return fmt.Errorf("invalid anime/episode")
}
t, err := normalizeSkipType(skipType)
if err != nil {
return err
}
if !(startTime >= 0) || !(endTime > startTime) {
return fmt.Errorf("invalid interval")
}
if endTime-startTime < 5 || endTime-startTime > 10*60 {
return fmt.Errorf("interval duration out of range")
}
return s.repo.UpsertSkipSegmentOverride(ctx, db.SkipSegmentOverrideRow{
ID: uuid.New().String(),
UserID: userID,
AnimeID: animeID,
Episode: int64(episode),
SkipType: t,
StartTime: startTime,
EndTime: endTime,
})
}
func (s *playbackService) fetchSkipSegments(ctx context.Context, userID string, malID int, episode string) []domain.SkipSegment {
if malID <= 0 || strings.TrimSpace(episode) == "" {
return []domain.SkipSegment{}
}
segments := s.fetchAniSkipSegments(ctx, malID, episode)
return s.applySkipSegmentOverrides(ctx, segments, userID, malID, episode)
}
func (s *playbackService) fetchAniSkipSegments(ctx context.Context, malID int, episode string) []domain.SkipSegment {
endpoint := fmt.Sprintf("https://api.aniskip.com/v1/skip-times/%s/%s?types=op&types=ed", url.PathEscape(strconv.Itoa(malID)), url.PathEscape(episode))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return nil
}
req.Header.Set("User-Agent", netutil.Generic)
resp, err := s.httpClient.Do(req)
if err != nil {
return nil
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return nil
}
body, err := io.ReadAll(io.LimitReader(resp.Body, netutil.KiB512))
if err != nil {
return nil
}
return parseAniSkipSegments(body)
}
func parseAniSkipSegments(body []byte) []domain.SkipSegment {
type resultItem struct {
SkipType string `json:"skip_type"`
Interval struct {
StartTime float64 `json:"start_time"`
EndTime float64 `json:"end_time"`
} `json:"interval"`
}
type apiResponse struct {
Found bool `json:"found"`
Result []resultItem `json:"results"`
}
var parsed apiResponse
if err := json.Unmarshal(body, &parsed); err != nil || !parsed.Found || len(parsed.Result) == 0 {
return nil
}
segments := make([]domain.SkipSegment, 0, len(parsed.Result))
for _, item := range parsed.Result {
segments = append(segments, domain.SkipSegment{
Type: normalizeSkipSegmentLabel(item.SkipType),
Start: item.Interval.StartTime,
End: item.Interval.EndTime,
Source: "aniskip",
})
}
return segments
}
func normalizeSkipSegmentLabel(skipType string) string {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op":
return "opening"
case "ed":
return "ending"
default:
return strings.ToLower(strings.TrimSpace(skipType))
}
}
func (s *playbackService) applySkipSegmentOverrides(ctx context.Context, segments []domain.SkipSegment, userID string, malID int, episode string) []domain.SkipSegment {
epNum, err := strconv.ParseInt(strings.TrimSpace(episode), 10, 64)
if userID == "" || err != nil || epNum <= 0 {
return segments
}
ok, err := s.repo.HasSkipSegmentOverrideTable(ctx)
if err != nil || !ok {
return segments
}
overrides, err := s.repo.ListSkipSegmentOverrides(ctx, userID, int64(malID), epNum)
if err != nil {
return segments
}
overrideByType := buildOverrideSegments(overrides)
if len(overrideByType) == 0 {
return segments
}
return mergeSkipSegments(segments, overrideByType)
}
func buildOverrideSegments(overrides []db.SkipSegmentOverrideRow) map[string]domain.SkipSegment {
byType := make(map[string]domain.SkipSegment, len(overrides))
for _, override := range overrides {
skipType, ok := normalizeOverrideSkipType(override.SkipType)
if !ok {
continue
}
byType[skipType] = domain.SkipSegment{
Type: skipType,
Start: override.StartTime,
End: override.EndTime,
Source: "override",
}
}
return byType
}
func normalizeOverrideSkipType(skipType string) (string, bool) {
switch strings.ToLower(strings.TrimSpace(skipType)) {
case "op", "opening", "intro":
return "opening", true
case "ed", "ending", "outro":
return "ending", true
default:
return "", false
}
}
func mergeSkipSegments(segments []domain.SkipSegment, overrides map[string]domain.SkipSegment) []domain.SkipSegment {
merged := make([]domain.SkipSegment, 0, len(segments)+len(overrides))
seen := make(map[string]bool, len(segments))
for _, segment := range segments {
if override, ok := overrides[segment.Type]; ok {
merged = append(merged, override)
} else {
merged = append(merged, segment)
}
seen[segment.Type] = true
}
for skipType, override := range overrides {
if !seen[skipType] {
merged = append(merged, override)
}
}
return merged
}

View File

@@ -0,0 +1,228 @@
package playback
import (
"context"
"fmt"
"sort"
"strconv"
"strings"
"mal/integrations/jikan"
"mal/internal/domain"
)
func (s *playbackService) BuildWatchData(ctx context.Context, animeID int, titleCandidates []string, episode string, mode string, userID string) (domain.WatchPageData, error) {
anime, err := s.jikan.GetAnimeByID(ctx, animeID)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch anime: %w", err)
}
animeData := domain.Anime{Anime: anime}
s.ensureAnimeRow(ctx, animeData)
searchTitles := buildSearchTitles(animeData, titleCandidates)
canonicalEpisodes, err := s.episodes.GetCanonicalEpisodes(ctx, animeData, false)
if err != nil {
return domain.WatchPageData{}, fmt.Errorf("failed to fetch episodes: %w", err)
}
mode, modeSwitchedFrom := resolveMode(episode, mode, canonicalEpisodes.Episodes)
modeSources, result, resolvedMode, resolvedModeSwitchedFrom := s.resolveModeSources(ctx, animeID, searchTitles, episode, mode)
if resolvedMode != "" {
mode = resolvedMode
}
if resolvedModeSwitchedFrom != "" {
modeSwitchedFrom = resolvedModeSwitchedFrom
}
if len(modeSources) == 0 {
return domain.WatchPageData{}, fmt.Errorf("no streams found")
}
if result == nil {
return domain.WatchPageData{}, fmt.Errorf("no streams found for mode %s", mode)
}
startTime, watchlistStatus, watchlistIDs := s.loadWatchProgress(ctx, userID, animeID, anime.Episodes, episode)
go s.warmStreamURL(result.URL, result.Referer)
seasons := s.loadSeasons(ctx, animeID)
segments := s.fetchSkipSegments(ctx, userID, animeID, episode)
watchData := buildWatchDataPayload(animeData, animeID, episode, startTime, canonicalEpisodes.Episodes, modeSources, mode, modeSwitchedFrom, segments)
return buildWatchPageData(animeData, canonicalEpisodes.Episodes, episode, watchlistStatus, watchlistIDs, seasons, watchData), nil
}
func buildWatchDataPayload(anime domain.Anime, animeID int, episode string, startTime float64, episodes []domain.CanonicalEpisode, modeSources map[string]domain.ModeSource, mode string, modeSwitchedFrom string, segments []domain.SkipSegment) domain.WatchData {
return domain.WatchData{
MalID: animeID,
Title: anime.DisplayTitle(),
CurrentEpisode: episode,
StartTimeSeconds: startTime,
Episodes: episodes,
Providers: []domain.ProviderData{{Streams: []domain.ProviderStream{{
Name: "Primary",
Quality: "Auto",
MalID: animeID,
IsCurrent: true,
}}}},
ModeSources: modeSources,
InitialMode: mode,
ModeSwitchedFrom: modeSwitchedFrom,
AvailableModes: availableModes(modeSources),
Segments: segments,
Airing: anime.Airing,
}
}
func buildWatchPageData(anime domain.Anime, episodes []domain.CanonicalEpisode, episode string, watchlistStatus string, watchlistIDs []int64, seasons []domain.SeasonEntry, watchData domain.WatchData) domain.WatchPageData {
return domain.WatchPageData{
WatchData: watchData,
Anime: anime,
Episodes: episodes,
CurrentEpID: episode,
WatchlistStatus: watchlistStatus,
WatchlistIDs: watchlistIDs,
Seasons: seasons,
}
}
func buildSearchTitles(anime domain.Anime, titleCandidates []string) []string {
seen := map[string]struct{}{}
out := make([]string, 0, 3+len(anime.TitleSynonyms)+len(titleCandidates))
appendTitle := func(title string) {
title = strings.TrimSpace(title)
if title == "" {
return
}
if _, ok := seen[title]; ok {
return
}
seen[title] = struct{}{}
out = append(out, title)
}
appendTitle(anime.Title)
appendTitle(anime.TitleEnglish)
appendTitle(anime.TitleJapanese)
for _, syn := range anime.TitleSynonyms {
appendTitle(syn)
}
for _, candidate := range titleCandidates {
appendTitle(candidate)
}
return out
}
func resolveMode(episode string, requestedMode string, episodes []domain.CanonicalEpisode) (string, string) {
if requestedMode != "dub" {
return requestedMode, ""
}
epNum, err := strconv.Atoi(episode)
if err != nil {
return requestedMode, ""
}
for _, ep := range episodes {
if ep.Number == epNum && !ep.HasDub && ep.HasSub {
return "sub", requestedMode
}
}
return requestedMode, ""
}
func (s *playbackService) resolveModeSources(ctx context.Context, animeID int, searchTitles []string, episode string, requestedMode string) (map[string]domain.ModeSource, *domain.StreamResult, string, string) {
if res := s.resolveStreamResult(ctx, animeID, searchTitles, episode, requestedMode); res != nil {
return map[string]domain.ModeSource{
requestedMode: s.buildModeSource(res),
}, res, requestedMode, ""
}
for _, fallbackMode := range fallbackModes(requestedMode) {
res := s.resolveStreamResult(ctx, animeID, searchTitles, episode, fallbackMode)
if res == nil {
continue
}
return map[string]domain.ModeSource{
fallbackMode: s.buildModeSource(res),
}, res, fallbackMode, requestedMode
}
return map[string]domain.ModeSource{}, nil, requestedMode, ""
}
func (s *playbackService) resolveStreamResult(ctx context.Context, animeID int, searchTitles []string, episode string, mode string) *domain.StreamResult {
for _, p := range s.providers {
res, err := p.GetStreams(ctx, animeID, searchTitles, episode, mode)
if err == nil && res != nil {
return res
}
}
return nil
}
func (s *playbackService) buildModeSource(res *domain.StreamResult) domain.ModeSource {
subtitles := make([]domain.SubtitleItem, 0, len(res.Subtitles))
for _, sub := range res.Subtitles {
token, _ := s.SignProxyToken(sub.URL, res.Referer, "subtitle")
subtitles = append(subtitles, domain.SubtitleItem{
Lang: sub.Label,
Token: token,
})
}
streamToken, _ := s.SignProxyToken(res.URL, res.Referer, "stream")
return domain.ModeSource{
Token: streamToken,
Type: res.Type,
Subtitles: subtitles,
}
}
func (s *playbackService) loadSeasons(ctx context.Context, animeID int) []domain.SeasonEntry {
relations, _ := s.jikan.GetFullRelations(ctx, animeID, jikan.WatchOrderModeMain)
seasons := make([]domain.SeasonEntry, 0, len(relations))
tvCounter := 1
for _, rel := range relations {
animeType := strings.ToLower(rel.Anime.Type)
if animeType != "tv" && animeType != "movie" {
continue
}
season := domain.SeasonEntry{
MalID: rel.Anime.MalID,
Title: rel.Anime.DisplayTitle(),
Prefix: rel.Relation,
IsCurrent: rel.IsCurrent,
}
if rel.Relation == "TV" {
season.Prefix = fmt.Sprintf("S%d", tvCounter)
tvCounter++
}
seasons = append(seasons, season)
}
return seasons
}
func availableModes(modeSources map[string]domain.ModeSource) []string {
modes := make([]string, 0, len(modeSources))
for mode := range modeSources {
modes = append(modes, mode)
}
sort.Strings(modes)
return modes
}
func fallbackModes(requestedMode string) []string {
switch requestedMode {
case "sub":
return []string{"dub"}
case "dub":
return []string{"sub"}
default:
return []string{"sub", "dub"}
}
}

View File

@@ -0,0 +1,35 @@
package playback
import (
"testing"
)
func TestFallbackModes(t *testing.T) {
t.Parallel()
tests := []struct {
name string
mode string
want []string
}{
{name: "sub falls back to dub", mode: "sub", want: []string{"dub"}},
{name: "dub falls back to sub", mode: "dub", want: []string{"sub"}},
{name: "unknown tries both canonical modes", mode: "raw", want: []string{"sub", "dub"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := fallbackModes(tt.mode)
if len(got) != len(tt.want) {
t.Fatalf("len(got) = %d, want %d", len(got), len(tt.want))
}
for i, want := range tt.want {
if got[i] != want {
t.Fatalf("got[%d] = %q, want %q", i, got[i], want)
}
}
})
}
}

View File

@@ -8,10 +8,6 @@ import (
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
) )
func CORSMiddleware() gin.HandlerFunc {
return CORSMiddlewareWithConfig(config.Config{})
}
func CORSMiddlewareWithConfig(cfg config.Config) gin.HandlerFunc { func CORSMiddlewareWithConfig(cfg config.Config) gin.HandlerFunc {
allowAll := cfg.CORSAllowAll allowAll := cfg.CORSAllowAll
return func(c *gin.Context) { return func(c *gin.Context) {

View File

@@ -31,23 +31,40 @@ func RequestLogger(metrics *observability.Metrics) gin.HandlerFunc {
level = observability.LogLevelWarn level = observability.LogLevelWarn
} }
observability.LogJSON( fields := map[string]any{
"client_ip": c.ClientIP(),
"duration_ms": float64(duration.Microseconds()) / 1000,
"method": c.Request.Method,
"path": path,
"request_id": c.Writer.Header().Get(requestIDHeader),
"status": status,
}
privateErrors := c.Errors.ByType(gin.ErrorTypePrivate)
var logErr error
if len(privateErrors) > 0 {
logErr = privateErrors.Last().Err
}
if route != path {
fields["route"] = route
}
if query != "" {
fields["query"] = query
}
if size := c.Writer.Size(); size >= 0 {
fields["bytes"] = size
}
if errors := privateErrors.String(); errors != "" {
fields["errors"] = errors
}
observability.LogContext(
c.Request.Context(),
level, level,
"http_request", "http_request",
"http", "http",
"", c.Request.Method+" "+path,
map[string]any{ fields,
"method": c.Request.Method, logErr,
"route": route,
"path": path,
"query": query,
"status": status,
"duration_ms": float64(duration.Microseconds()) / 1000,
"bytes": c.Writer.Size(),
"client_ip": c.ClientIP(),
"errors": c.Errors.ByType(gin.ErrorTypePrivate).String(),
},
nil,
) )
} }
} }

View File

@@ -0,0 +1,30 @@
package server
import (
"mal/internal/observability"
"strings"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
)
const requestIDHeader = "X-Request-ID"
func RequestContextMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
requestID := strings.TrimSpace(c.GetHeader(requestIDHeader))
if requestID == "" {
requestID = uuid.NewString()
}
path := c.Request.URL.Path
route := c.FullPath()
if route == "" {
route = path
}
c.Writer.Header().Set(requestIDHeader, requestID)
c.Request = c.Request.WithContext(observability.WithRequestContext(c.Request.Context(), requestID, path, route))
c.Next()
}
}

View File

@@ -27,7 +27,18 @@ func RespondError(c *gin.Context, status int, event string, component string, me
if status >= http.StatusInternalServerError { if status >= http.StatusInternalServerError {
level = observability.LogLevelError level = observability.LogLevelError
} }
observability.LogJSON(level, event, component, "", fields, err) if fields == nil {
fields = make(map[string]any, 2)
}
if _, exists := fields["request_path"]; !exists {
fields["request_path"] = c.Request.URL.Path
}
if route := c.FullPath(); route != "" && route != c.Request.URL.Path {
if _, exists := fields["request_route"]; !exists {
fields["request_route"] = route
}
}
observability.LogContext(c.Request.Context(), level, event, component, "", fields, err)
RespondHTMLOrJSONError(c, status, message) RespondHTMLOrJSONError(c, status, message)
} }

View File

@@ -27,7 +27,7 @@ func ProvideRouter(cfg config.Config, htmlRender render.HTMLRender, metrics *obs
gin.SetMode(cfg.GinMode) gin.SetMode(cfg.GinMode)
} }
r := gin.New() r := gin.New()
r.Use(CORSMiddlewareWithConfig(cfg), audit.ContextMiddleware(), RequestLogger(metrics), gin.Recovery()) r.Use(CORSMiddlewareWithConfig(cfg), RequestContextMiddleware(), audit.ContextMiddleware(), RequestLogger(metrics), gin.Recovery())
r.Static("/static", "./static") r.Static("/static", "./static")
r.Static("/dist", "./dist") r.Static("/dist", "./dist")
r.GET("/metrics", gin.WrapH(metrics.Handler())) r.GET("/metrics", gin.WrapH(metrics.Handler()))

View File

@@ -2,6 +2,7 @@ package server
import ( import (
"bytes" "bytes"
"context"
"io" "io"
"log" "log"
"mal/internal/observability" "mal/internal/observability"
@@ -43,12 +44,13 @@ func TestRequestLoggerUsesMatchedRoute(t *testing.T) {
defer log.SetOutput(previousOutput) defer log.SetOutput(previousOutput)
router := gin.New() router := gin.New()
router.Use(RequestContextMiddleware())
router.Use(RequestLogger(observability.NewMetrics())) router.Use(RequestLogger(observability.NewMetrics()))
router.GET("/anime/:id", func(c *gin.Context) { router.GET("/anime/:id", func(c *gin.Context) {
c.String(http.StatusOK, "ok") c.String(http.StatusOK, "ok")
}) })
req := httptest.NewRequest(http.MethodGet, "/anime/1?section=characters", nil) req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/anime/1?section=characters", nil)
rec := httptest.NewRecorder() rec := httptest.NewRecorder()
router.ServeHTTP(rec, req) router.ServeHTTP(rec, req)
@@ -58,13 +60,54 @@ func TestRequestLoggerUsesMatchedRoute(t *testing.T) {
} }
logLine := string(output) logLine := string(output)
if !strings.Contains(logLine, `"event":"http_request"`) { if !strings.Contains(logLine, " INFO http 200 GET /anime/1") {
t.Fatalf("log line missing event: %s", logLine) t.Fatalf("log line missing compact http summary: %s", logLine)
} }
if !strings.Contains(logLine, `"route":"/anime/:id"`) { if !strings.Contains(logLine, " route=/anime/:id") {
t.Fatalf("log line missing route: %s", logLine) t.Fatalf("log line missing route: %s", logLine)
} }
if !strings.Contains(logLine, `"status":200`) { if !strings.Contains(logLine, " request_id=") {
t.Fatalf("log line missing status: %s", logLine) t.Fatalf("log line missing request id: %s", logLine)
}
if strings.Contains(logLine, `"GET /anime/1"`) {
t.Fatalf("log line should not duplicate request summary: %s", logLine)
}
if rec.Header().Get(requestIDHeader) == "" {
t.Fatalf("expected %s response header to be set", requestIDHeader)
}
}
func TestRespondErrorIncludesRequestContext(t *testing.T) {
gin.SetMode(gin.TestMode)
var logs bytes.Buffer
previousOutput := log.Writer()
log.SetOutput(&logs)
defer log.SetOutput(previousOutput)
router := gin.New()
router.Use(RequestContextMiddleware())
router.GET("/anime/:id", func(c *gin.Context) {
RespondError(c, http.StatusInternalServerError, "anime_lookup_failed", "anime", "failed", nil, context.DeadlineExceeded)
})
req := httptest.NewRequestWithContext(context.Background(), http.MethodGet, "/anime/1", nil)
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
output, err := io.ReadAll(&logs)
if err != nil {
t.Fatalf("read logs: %v", err)
}
logLine := string(output)
if !strings.Contains(logLine, " request_id=") {
t.Fatalf("log line missing request id: %s", logLine)
}
if !strings.Contains(logLine, " request_path=/anime/1") {
t.Fatalf("log line missing request path: %s", logLine)
}
if !strings.Contains(logLine, " request_route=/anime/:id") {
t.Fatalf("log line missing request route: %s", logLine)
} }
} }

View File

@@ -17,14 +17,13 @@ test:
go test ./... go test ./...
build-go: build-go:
go build -o server ./cmd/server @go build -o server ./cmd/server
build-css: build-css:
bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css @bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css
build-ts: build-ts:
bun build ./static/player/main.ts --outdir ./dist/static/player --target browser --splitting @bun ./scripts/build-ts.ts
bun build ./static/*.ts --outdir ./dist/static --target browser --root ./static --entry-naming "[name].js"
build: build-go build-css build-ts build: build-go build-css build-ts
@@ -37,7 +36,7 @@ install-hooks:
bunx lefthook install bunx lefthook install
dev: build dev: build
./server @./server
clean: clean:
rm -rf dist/* rm -rf dist/*

View File

@@ -2,17 +2,42 @@
"$schema": "https://json.schemastore.org/lefthook.json", "$schema": "https://json.schemastore.org/lefthook.json",
"pre-commit": "pre-commit":
{ {
"fail_on_changes": "always",
"fail_on_changes_diff": true,
"commands": "commands":
{ {
"format": { "run": "bunx oxfmt" }, "format":
{
"glob": "*.{ts,js,tsx,jsx,css,json,html}",
"run": "bunx oxfmt --check {staged_files}",
},
"lint:ts": "lint:ts":
{ "run": "bunx oxlint --ignore-path .oxlintignore static --max-warnings 0 --fix" }, {
"go-fmt": { "run": "go fmt ./..." }, "glob": "*.{ts,js,tsx,jsx}",
"go-lint": { "run": "bun run lint:go" }, "run": "bunx oxlint --ignore-path .oxlintignore {staged_files} --max-warnings 0 --tsconfig ./tsconfig.json --type-aware",
"go-test": { "run": "go test ./..." }, },
"ts-typecheck": { "run": "bunx tsc -p tsconfig.json --noEmit" }, "go-fmt":
"build-assets": { "run": "bun run build:assets" }, {
"go-build": { "run": "go build -o server ./cmd/server" }, "glob": "*.go",
"run": 'files=$(gofmt -l {staged_files}); test -z "$files" || (printf "go files need formatting:\n%s\n" "$files"; exit 1)',
},
"go-lint":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} golangci-lint run --new-from-rev=HEAD ./{}',
},
"go-test":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} go test -count=1 ./{}',
},
"ts-typecheck": { "glob": "*.ts", "run": "bunx tsc -p tsconfig.json --noEmit" },
"build-assets": { "glob": "*.{ts,css}", "run": "bun run build:assets" },
"go-build":
{
"glob": "*.go",
"run": 'printf "%s\n" {staged_files} | xargs -n1 dirname | sort -u | xargs -I{} go build -o /dev/null ./{}',
},
}, },
}, },
} }

View File

@@ -3,9 +3,9 @@
"private": true, "private": true,
"type": "module", "type": "module",
"scripts": { "scripts": {
"build:css": "bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css", "build:css": "bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css",
"watch:css": "bunx @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css --watch", "watch:css": "bunx --bun @tailwindcss/cli -i ./static/assets/style.css -o ./dist/tailwind.css --watch",
"build:ts": "bun build ./static/player/main.ts --outdir ./dist/static/player --target browser --splitting && bun build ./static/*.ts --outdir ./dist/static --target browser --root ./static --entry-naming \"[name].js\" && cp ./node_modules/htmx.org/dist/htmx.min.js ./dist/static/htmx-lib.js", "build:ts": "bun ./scripts/build-ts.ts",
"typecheck": "bunx tsc -p tsconfig.json --noEmit", "typecheck": "bunx tsc -p tsconfig.json --noEmit",
"build:assets": "bun run build:css && bun run build:ts", "build:assets": "bun run build:css && bun run build:ts",
"format": "bunx oxfmt", "format": "bunx oxfmt",
@@ -16,6 +16,7 @@
"lint:go": "golangci-lint run ./..." "lint:go": "golangci-lint run ./..."
}, },
"dependencies": { "dependencies": {
"hls.js": "^1.6.16",
"htmx.org": "1.9.12" "htmx.org": "1.9.12"
}, },
"devDependencies": { "devDependencies": {

View File

@@ -53,12 +53,12 @@ func Post[T any](ctx context.Context, httpClient *http.Client, url string, query
} }
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
max := opts.BodyMax bodyMax := opts.BodyMax
if max <= 0 { if bodyMax <= 0 {
max = 2 << 20 bodyMax = 2 << 20
} }
respBody, err := io.ReadAll(io.LimitReader(resp.Body, max)) respBody, err := io.ReadAll(io.LimitReader(resp.Body, bodyMax))
if err != nil { if err != nil {
return zero, fmt.Errorf("graphql: read response: %w", err) return zero, fmt.Errorf("graphql: read response: %w", err)
} }

View File

@@ -9,13 +9,23 @@ import (
"github.com/PuerkitoBio/goquery" "github.com/PuerkitoBio/goquery"
) )
func responseURL(response *http.Response, fallbackRequest *http.Request) string {
if response != nil && response.Request != nil && response.Request.URL != nil {
return response.Request.URL.String()
}
if fallbackRequest != nil && fallbackRequest.URL != nil {
return fallbackRequest.URL.String()
}
return ""
}
func FetchHTMLDocument( func FetchHTMLDocument(
ctx context.Context, ctx context.Context,
httpClient *http.Client, httpClient *http.Client,
url string, url string,
prepareRequest func(*http.Request), prepareRequest func(*http.Request),
buildStatusError func(*http.Response, []byte) error, buildStatusError func(*http.Response, []byte) error,
) (*goquery.Document, *http.Response, error) { ) (*goquery.Document, string, error) {
client := httpClient client := httpClient
if client == nil { if client == nil {
client = http.DefaultClient client = http.DefaultClient
@@ -23,7 +33,7 @@ func FetchHTMLDocument(
request, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) request, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to create request: %w", err) return nil, "", fmt.Errorf("failed to create request: %w", err)
} }
if prepareRequest != nil { if prepareRequest != nil {
prepareRequest(request) prepareRequest(request)
@@ -31,19 +41,19 @@ func FetchHTMLDocument(
response, err := client.Do(request) response, err := client.Do(request)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("request failed: %w", err) return nil, "", fmt.Errorf("request failed: %w", err)
} }
defer func() { _ = response.Body.Close() }() defer func() { _ = response.Body.Close() }()
if response.StatusCode != http.StatusOK { if response.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(response.Body, Bytes512)) body, _ := io.ReadAll(io.LimitReader(response.Body, Bytes512))
return nil, response, buildStatusError(response, body) return nil, responseURL(response, request), buildStatusError(response, body)
} }
document, err := goquery.NewDocumentFromReader(response.Body) document, err := goquery.NewDocumentFromReader(response.Body)
if err != nil { if err != nil {
return nil, response, fmt.Errorf("failed to parse html: %w", err) return nil, responseURL(response, request), fmt.Errorf("failed to parse html: %w", err)
} }
return document, response, nil return document, responseURL(response, request), nil
} }

41
pkg/net/document_test.go Normal file
View File

@@ -0,0 +1,41 @@
package netutil
import (
"context"
"io"
"net/http"
"strings"
"testing"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) {
return f(request)
}
func TestFetchHTMLDocumentFallsBackToOriginalURLWhenResponseRequestMissing(t *testing.T) {
client := &http.Client{
Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader("<!doctype html><html><body><main>ok</main></body></html>")),
}, nil
}),
}
url := "https://example.test/watch-order"
document, finalURL, err := FetchHTMLDocument(context.Background(), client, url, nil, nil)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if finalURL != url {
t.Fatalf("expected final url %q, got %q", url, finalURL)
}
if got := strings.TrimSpace(document.Find("main").Text()); got != "ok" {
t.Fatalf("expected document text ok, got %q", got)
}
}

91
scripts/build-ts.ts Normal file
View File

@@ -0,0 +1,91 @@
import { copyFile } from "node:fs/promises";
import { readdirSync } from "node:fs";
import { spawnSync } from "node:child_process";
type BuildStep = {
name: string;
command: string[];
};
const steps: BuildStep[] = [
{
name: "player",
command: [
"bun",
"build",
"./static/player/main.ts",
"--outdir",
"./dist/static/player",
"--target",
"browser",
"--splitting",
],
},
];
const startedAt = performance.now();
main().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
console.error(`ts build failed: ${message}`);
process.exit(1);
});
async function main(): Promise<void> {
const appEntries = readdirSync("./static", { withFileTypes: true })
.filter((entry) => entry.isFile() && entry.name.endsWith(".ts"))
.map((entry) => `./static/${entry.name}`)
.sort();
steps.push({
name: "app",
command: [
"bun",
"build",
...appEntries,
"--outdir",
"./dist/static",
"--target",
"browser",
"--root",
"./static",
"--entry-naming",
"[name].js",
],
});
for (const step of steps) {
const result = spawnSync(step.command[0], step.command.slice(1), {
stdio: "pipe",
});
if (result.status !== 0) {
const detail = summarizeFailure(result.stderr, result.stdout);
console.error(`ts build failed at ${step.name}${detail === "" ? "" : `: ${detail}`}`);
process.exit(result.status ?? 1);
}
}
await copyFile("./node_modules/htmx.org/dist/htmx.min.js", "./dist/static/htmx-lib.js");
const playerEntries = 1;
const totalEntries = playerEntries + appEntries.length;
const elapsedMs = Math.round(performance.now() - startedAt);
console.log(`ts build ok (${totalEntries} entries, ${elapsedMs}ms)`);
}
function summarizeFailure(stderr: Uint8Array, stdout: Uint8Array): string {
const combined =
`${Buffer.from(stderr).toString("utf8")}${Buffer.from(stdout).toString("utf8")}`.trim();
if (combined === "") {
return "";
}
const lines = combined
.split(/\r?\n/)
.map((line) => line.trim())
.filter((line) => line !== "");
return lines[lines.length - 1] ?? "";
}

View File

@@ -2,15 +2,12 @@ import "./theme";
import "./toast"; import "./toast";
import "./htmx"; import "./htmx";
import "./dropdown"; import "./dropdown";
import "./discover";
import "./anime"; import "./anime";
import "./timezone"; import "./timezone";
import "./search"; import "./search";
import "./sort_filter"; import "./sort_filter";
import "./dedupe"; import "./dedupe";
import "./shell";
import "./watchlist"; import "./watchlist";
import "./top_pick_carousel"; import "./top_pick_carousel";
import "./continue_watching_carousel"; import "./continue_watching_carousel";
import "./login"; import "./login";
import "./schedule";

BIN
static/assets/app-icon.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>

Before

Width:  |  Height:  |  Size: 608 B

View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>

Before

Width:  |  Height:  |  Size: 608 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.3 KiB

View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>

Before

Width:  |  Height:  |  Size: 608 B

View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>

Before

Width:  |  Height:  |  Size: 608 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

View File

@@ -1,18 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
</defs>
<!-- Background square (for mobile home screen / maskable) -->
<rect width="100" height="100" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</svg>

Before

Width:  |  Height:  |  Size: 608 B

BIN
static/assets/favicon.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 113 KiB

View File

@@ -1,23 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100">
<defs>
<radialGradient id="bg" cx="35%" cy="35%" r="75%">
<stop offset="0%" style="stop-color: var(--accent, #0466c8)" />
<stop offset="100%" style="stop-color: var(--accent-dark, #1d4ed8)" />
</radialGradient>
<clipPath id="clip">
<circle cx="50" cy="50" r="45" />
</clipPath>
</defs>
<!-- Base -->
<circle cx="50" cy="50" r="45" fill="url(#bg)" />
<!-- Crescent moon cutout -->
<g clip-path="url(#clip)">
<path
d="M70 50a25 25 0 1 1 -25 -25 20 20 0 1 0 25 25z"
fill="#FFF7ED"
transform="translate(-2 -2)"
/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 685 B

BIN
static/assets/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 392 KiB

Some files were not shown because too many files have changed in this diff Show More