Commit Graph

25 Commits

Author SHA1 Message Date
cce5f2463d still use redis for cache 2024-05-14 17:16:26 +08:00
04ba62977e fix rebase conflicts 2024-05-14 17:14:41 +08:00
e093c9e096 fix comment 2024-05-14 17:14:41 +08:00
3e925e0193 update comment 2024-05-14 17:14:41 +08:00
5bd157ca25 hash url as the key 2024-05-14 17:14:41 +08:00
7a0b2f3d33 upload file only not exists 2024-05-14 17:14:41 +08:00
9286174ec7 upload and download original content from GCS 2024-05-14 17:14:40 +08:00
7634ed667f capture total time of fetching a page 2024-05-13 17:01:52 +08:00
f64bd4700f update analytic event details 2024-05-13 15:18:04 +08:00
a924c8448b capture content-fetch success and error events 2024-05-13 14:55:48 +08:00
927394e07c fix: save url operation is delayed 2024-02-21 17:54:08 +08:00
86f4553dd1 fix: duplicate key value violates unique constraint "library_item_pkey" 2024-02-07 20:41:34 +08:00
fd80724d79 fallback to guid if link of rss item is not available 2024-01-26 22:09:53 +08:00
e4e07c8acf cache fetched content for 24 hours 2024-01-26 13:01:53 +08:00
5e239d2568 run readability in save-page instead of puppeteer 2024-01-25 16:30:59 +08:00
fd0bb37d3a make title optional for cache 2024-01-23 16:54:35 +08:00
356ad38678 reduce logs 2024-01-19 18:01:09 +08:00
8acdea5002 set cache if not exists 2024-01-19 17:14:49 +08:00
51d5166321 get cached content from redis when saving pages 2024-01-19 17:06:37 +08:00
699d38609b cache fetch result in redis 2024-01-19 16:58:24 +08:00
d4f49dceb6 remove job id 2024-01-18 18:48:46 +08:00
a6f0e2f2d9 add save page job processor 2024-01-18 18:48:46 +08:00
43df5d30a8 prioritized jobs 2024-01-18 18:48:46 +08:00
0015a946b2 create batch save page jobs after fetching content 2024-01-18 18:48:46 +08:00
d9feb740cb convert content-fetch to typescript 2024-01-18 18:48:46 +08:00