116 Commits

Author	SHA1	Message	Date
Hongbo Wu	ce19218e15	get queue priority from a table	2024-02-21 22:01:04 +08:00
Hongbo Wu	927394e07c	fix: save url operation is delayed	2024-02-21 17:54:08 +08:00
Hongbo Wu	485caec803	append job version to the job id	2024-02-07 18:42:10 +08:00
Hongbo Wu	ef6c6349de	add remove_on_complete and remove_on_fail to the unique job	2024-02-07 18:03:16 +08:00
Hongbo Wu	468f835f9e	Merge pull request #3502 from omnivore-app/fix/bullmq-job-retry-backoff ... increase the seed valud of backoff delay to 2 seconds	2024-02-06 15:09:40 +08:00
Hongbo Wu	a939180dc9	fix: dedupe update-db job by using library_item_id as job_id	2024-02-06 14:10:53 +08:00
Hongbo Wu	53f63b90d7	increase the seed valud of backoff delay to 2 seconds	2024-02-05 16:26:05 +08:00
Jackson Harper	0a38cd6422	More attempts now that lock timeouts are lower	2024-02-03 19:08:13 +08:00
Hongbo Wu	1742ffda24	change priority to 5	2024-02-02 16:18:41 +08:00
Hongbo Wu	25e374f6ff	fix: replace webhook endpoint with a bullmq job	2024-02-02 16:18:40 +08:00
Hongbo Wu	26b7103f0b	improve batch update	2024-02-02 13:13:18 +08:00
Hongbo Wu	1bccf33320	add job_id and parent for tracking	2024-02-02 10:50:07 +08:00
Hongbo Wu	abaf726044	revert the change of priority	2024-02-01 13:22:06 +08:00
Hongbo Wu	19fe60d27a	change the update-db job priority to 5 and use exponential backoff strategy for retrying	2024-02-01 13:14:01 +08:00
Hongbo Wu	c3276fdaf1	Merge pull request #3466 from omnivore-app/fix/rewrite-trigger-functions-in-db ... create a job for db trigger	2024-01-31 19:47:45 +08:00
Hongbo Wu	1b8111b907	wait until all jobs are done in test	2024-01-31 19:10:57 +08:00
Hongbo Wu	5646413b59	add a job for updating highlight annotations in library_item	2024-01-31 14:15:33 +08:00
Hongbo Wu	5ed35bbeff	create a job for update labels in library item	2024-01-31 13:32:37 +08:00
Hongbo Wu	ddddf8234e	create a job for db trigger	2024-01-30 23:53:05 +08:00
Hongbo Wu	36d2c21466	remove thumbnail job on complete or failure	2024-01-30 18:24:48 +08:00
Hongbo Wu	2eedba1522	fix: assign the rule job with priority 1 and do not retry	2024-01-30 18:14:34 +08:00
Hongbo Wu	d5d581fc54	remove redundant logs	2024-01-29 14:32:43 +08:00
Hongbo Wu	30fef30384	resolve conflicts	2024-01-26 20:32:56 +08:00
Hongbo Wu	e03895c8b7	enqueue trigger rule job once page is created or updated	2024-01-26 17:46:17 +08:00
Hongbo Wu	52419c39da	resolve conflicts	2024-01-26 13:01:53 +08:00
Hongbo Wu	4c765378e3	do not retry if find_thumbnail job fails	2024-01-26 13:01:53 +08:00
Hongbo Wu	e54c691c09	create a job for finding thumbnail	2024-01-26 13:01:51 +08:00
Hongbo Wu	68a4fb298d	add refreshedAt to the gql schema	2024-01-24 12:03:26 +08:00
Hongbo Wu	8f5d4ae8a0	fix: catch rejected promise if it is in a try-catch block	2024-01-23 18:45:46 +08:00
Jackson Harper	2961f69f02	Add a refresh context to make debugging rss jobs easier	2024-01-20 09:47:46 +08:00
Jackson Harper	fb28d7b3d1	Enqueue new feeds at front	2024-01-18 14:34:07 +08:00
Jackson Harper	9e6b749604	Update error message	2024-01-17 16:21:42 +08:00
Jackson Harper	e1e2f86219	Remove duplicated error	2024-01-17 16:20:42 +08:00
Jackson Harper	7a925f3fdc	Clean up redis code, remove connection close in job	2024-01-17 11:14:59 +08:00
Jackson Harper	50b48a4225	Separate worker and client code for redis connections	2024-01-16 23:21:48 +08:00
Jackson Harper	e050864a91	Add more debugging	2024-01-16 21:30:17 +08:00
Jackson Harper	62fdb866a8	Add some extra logging	2024-01-16 21:30:17 +08:00
Jackson Harper	5986bd0a68	Change around redis initialization so we can pull in secrets from secret manager async on startup	2024-01-16 21:30:15 +08:00
Jackson Harper	fc25d6d615	Enqueue rss-refresh-feed jobs in bull	2024-01-16 21:25:31 +08:00
Hongbo Wu	97735d0633	make existing newsletters go to inbox folder	2023-12-18 16:08:36 +08:00
Hongbo Wu	f2c468f906	allow setting fetchContent and folder when subscribe to a feed	2023-12-14 16:32:48 +08:00
Hongbo Wu	3759e10615	fix feed url in pdf file not saved	2023-12-08 11:29:02 +08:00
Hongbo Wu	824b8187af	use system token in integration handler	2023-12-04 15:38:08 +08:00
Hongbo Wu	fd781644f1	feat: fetch content for rss feed items in following folder	2023-11-23 18:03:25 +08:00
Hongbo Wu	9dfc5f856d	rename variables	2023-11-09 19:00:57 +08:00
Hongbo Wu	9b0b378c39	check is_fetching_content and only fetch content if required for rss feed items	2023-11-08 18:16:48 +08:00
Hongbo Wu	8df6fb3ae2	import unarchived items by default	2023-10-30 16:22:09 +08:00
Hongbo Wu	07eb97e7cc	remove unused code	2023-10-30 14:41:53 +08:00
Hongbo Wu	58d5eb6bc0	create an API to create a cloud task for each active integration when the cronjob triggers	2023-10-30 14:41:52 +08:00
Hongbo Wu	1880f2ace4	Update rss handler to fetch a group of rss feeds in one task	2023-10-19 17:27:55 +08:00