Commit Graph

57 Commits

Author SHA1 Message Date
af7350bd31 fix: importer status update email not sent
* create omnivore-authorization header to contain the token
2023-06-28 11:00:40 +08:00
8ad937ed84 allow more labels format in the csv 2023-06-14 13:33:20 +08:00
42305ee0b4 use a separate queue for sending emails 2023-06-14 13:17:59 +08:00
dddd763ba2 fix graphql error by setting clientRequestId empty 2023-06-13 21:15:34 +08:00
27ff2856e2 remove client request id in import task which caused the pdf upload failure 2023-06-13 18:21:36 +08:00
4a438f45f5 add timeout and content length to the axios requests 2023-06-12 21:31:35 +08:00
63bddf13a1 set default metrics to zero 2023-06-05 18:16:41 +08:00
4cad18f51c fix taskId not being set in the cloud task payload 2023-06-05 17:52:36 +08:00
992a78e173 add dockerfile 2023-06-01 16:29:30 +08:00
8a89a47c16 fix: testcases 2023-06-01 16:11:35 +08:00
813a90e3c9 feat: send importer status update 2023-06-01 16:11:35 +08:00
fba8e0424d feat: add importer metrics collector 2023-06-01 16:11:35 +08:00
5f465112bf feat: update matter importer metrics 2023-06-01 16:11:35 +08:00
67014e68fd feat: send email notification once the import has finished 2023-06-01 16:11:35 +08:00
19facec4e2 feat: add importer metrics collector 2023-06-01 16:11:35 +08:00
f1f83a6891 Handle empty state columns in the import CSV files 2023-05-24 18:09:42 +08:00
2016665323 Set labels = undefined if pocket tags is empty array 2023-05-16 12:23:10 +08:00
e14080a05f Set labels = undefined if pocket tags is undefined 2023-05-16 11:04:22 +08:00
2423bff11c Fix parsing error on empty array of labels in the csv file 2023-05-16 10:27:55 +08:00
c03d59c3c9 Add test case for failures 2023-04-17 17:20:14 +08:00
a1e782e39d Fix tests 2023-04-17 17:14:40 +08:00
11538347af Fix incorrect successful and failed counter of imported urls in the importer 2023-04-17 10:24:46 +08:00
87d3f5b3d8 Add labels and state to savePage API 2023-04-12 16:50:11 +08:00
fcf08fc364 Add state and labels to savePageInput 2023-04-12 16:50:11 +08:00
52399b3702 Add state and labels to the cloud task 2023-04-12 16:50:11 +08:00
4b578bbebb Process labels and state in the csv file 2023-04-12 16:50:11 +08:00
46e371bc26 Remove more debug 2023-01-24 20:36:34 +08:00
1d81398bbd Improve logging on errors in the importer tool 2023-01-24 20:03:53 +08:00
b7abb5fa91 Use glob to find _matter_history file in archives
It looks like these have another directory within the archive, so
we have to search for the _matter_history file to find the full
archive content.
2023-01-24 14:14:04 +08:00
6845696f23 Handle exceptions from JSON.parse, in this case just return ok so the request isnt retried 2023-01-12 16:10:25 +08:00
6f588e2170 Create read stream after all early return checks 2023-01-12 15:48:54 +08:00
444d4a69eb i guess this was my first rodeo 2023-01-12 15:37:50 +08:00
c240f1d2c8 Make SavePage API call if we have content provided 2023-01-12 10:50:06 +08:00
ee8bad424b Fix typo 2023-01-11 23:19:59 +08:00
fa866f1d29 Make function more clear 2023-01-11 23:08:30 +08:00
0bc87deba0 Change order of ctx arguments 2023-01-11 22:55:04 +08:00
04375d6856 Add some extra debugging 2023-01-11 22:42:37 +08:00
6623d5a43a Handle pubsub event parsing 2023-01-11 20:59:41 +08:00
62db9da121 Parse pubsub message 2023-01-11 20:07:54 +08:00
154109bb9d Log headers 2023-01-11 19:46:12 +08:00
50e1dbda5a Add debug for Eventarc 2023-01-11 18:36:37 +08:00
dea4ffae33 Fixes for dockerizing the import-handler service 2023-01-11 18:10:22 +08:00
e31a806fcb Start http server to run in Cloud Run 2023-01-11 16:57:00 +08:00
e392af4800 Update matter import handler to use archives instead of just history files 2023-01-11 15:35:05 +08:00
592c7150a7 Handle case where import has a single entry 2023-01-06 12:49:29 +08:00
88b6fcfb5d Use cookie auth instead of header as GCP uses the Authorization header 2023-01-05 20:17:43 +08:00
0580f7a876 Fix the email endpoint for the import-handler 2023-01-05 17:58:26 +08:00
687d126bd3 Fix the task URLs, send an email if no URLs are parsed for import 2023-01-05 11:49:16 +08:00
f177a0d1db Remove unused type 2023-01-05 11:14:57 +08:00
37dc0336fa Set Authorization to authToken 2023-01-05 11:13:58 +08:00