|
|
9ee87e2e52
|
convert type of date in the csv to int before parsing date
|
2023-08-15 18:44:30 +08:00 |
|
|
|
e3654e8ae2
|
fix test
|
2023-08-14 16:38:24 +08:00 |
|
|
|
d25f19207b
|
add savedAt and publishedAt to the csv file
|
2023-08-14 13:49:06 +08:00 |
|
|
|
ba2d34c1af
|
validate csv file in the importer
|
2023-08-14 13:38:36 +08:00 |
|
|
|
792cf0b207
|
upgrade google cloud sdk
|
2023-08-11 13:55:29 +08:00 |
|
|
|
fa1ff9ba17
|
Upgrade node to 18.16
|
2023-07-24 15:55:11 +08:00 |
|
|
|
fc7a649566
|
limit import to 20k urls
|
2023-07-17 18:09:13 +08:00 |
|
|
|
af7350bd31
|
fix: importer status update email not sent
* create omnivore-authorization header to contain the token
|
2023-06-28 11:00:40 +08:00 |
|
|
|
8ad937ed84
|
allow more labels format in the csv
|
2023-06-14 13:33:20 +08:00 |
|
|
|
42305ee0b4
|
use a separate queue for sending emails
|
2023-06-14 13:17:59 +08:00 |
|
|
|
dddd763ba2
|
fix graphql error by setting clientRequestId empty
|
2023-06-13 21:15:34 +08:00 |
|
|
|
27ff2856e2
|
remove client request id in import task which caused the pdf upload failure
|
2023-06-13 18:21:36 +08:00 |
|
|
|
4a438f45f5
|
add timeout and content length to the axios requests
|
2023-06-12 21:31:35 +08:00 |
|
|
|
63bddf13a1
|
set default metrics to zero
|
2023-06-05 18:16:41 +08:00 |
|
|
|
4cad18f51c
|
fix taskId not being set in the cloud task payload
|
2023-06-05 17:52:36 +08:00 |
|
|
|
992a78e173
|
add dockerfile
|
2023-06-01 16:29:30 +08:00 |
|
|
|
8a89a47c16
|
fix: testcases
|
2023-06-01 16:11:35 +08:00 |
|
|
|
1073f88d69
|
fix: add start:collector command
|
2023-06-01 16:11:35 +08:00 |
|
|
|
813a90e3c9
|
feat: send importer status update
|
2023-06-01 16:11:35 +08:00 |
|
|
|
fba8e0424d
|
feat: add importer metrics collector
|
2023-06-01 16:11:35 +08:00 |
|
|
|
5f465112bf
|
feat: update matter importer metrics
|
2023-06-01 16:11:35 +08:00 |
|
|
|
67014e68fd
|
feat: send email notification once the import has finished
|
2023-06-01 16:11:35 +08:00 |
|
|
|
19facec4e2
|
feat: add importer metrics collector
|
2023-06-01 16:11:35 +08:00 |
|
|
|
c64476b44a
|
Add test for label parsing
|
2023-05-24 18:17:29 +08:00 |
|
|
|
9218e16415
|
Add test file
|
2023-05-24 18:10:30 +08:00 |
|
|
|
f1f83a6891
|
Handle empty state columns in the import CSV files
|
2023-05-24 18:09:42 +08:00 |
|
|
|
2016665323
|
Set labels = undefined if pocket tags is empty array
|
2023-05-16 12:23:10 +08:00 |
|
|
|
e14080a05f
|
Set labels = undefined if pocket tags is undefined
|
2023-05-16 11:04:22 +08:00 |
|
|
|
2423bff11c
|
Fix parsing error on empty array of labels in the csv file
|
2023-05-16 10:27:55 +08:00 |
|
|
|
c03d59c3c9
|
Add test case for failures
|
2023-04-17 17:20:14 +08:00 |
|
|
|
a1e782e39d
|
Fix tests
|
2023-04-17 17:14:40 +08:00 |
|
|
|
11538347af
|
Fix incorrect successful and failed counter of imported urls in the importer
|
2023-04-17 10:24:46 +08:00 |
|
|
|
87d3f5b3d8
|
Add labels and state to savePage API
|
2023-04-12 16:50:11 +08:00 |
|
|
|
fcf08fc364
|
Add state and labels to savePageInput
|
2023-04-12 16:50:11 +08:00 |
|
|
|
52399b3702
|
Add state and labels to the cloud task
|
2023-04-12 16:50:11 +08:00 |
|
|
|
4b578bbebb
|
Process labels and state in the csv file
|
2023-04-12 16:50:11 +08:00 |
|
|
|
46e371bc26
|
Remove more debug
|
2023-01-24 20:36:34 +08:00 |
|
|
|
1d81398bbd
|
Improve logging on errors in the importer tool
|
2023-01-24 20:03:53 +08:00 |
|
|
|
cba676c9de
|
Build fixes in import-handler
|
2023-01-24 14:55:00 +08:00 |
|
|
|
b7abb5fa91
|
Use glob to find _matter_history file in archives
It looks like these have another directory within the archive, so
we have to search for the _matter_history file to find the full
archive content.
|
2023-01-24 14:14:04 +08:00 |
|
|
|
d2e7f8e58c
|
Add .dockerignore for the import handler
|
2023-01-13 09:19:15 +08:00 |
|
|
|
6845696f23
|
Handle exceptions from JSON.parse, in this case just return ok so the request isnt retried
|
2023-01-12 16:10:25 +08:00 |
|
|
|
6f588e2170
|
Create read stream after all early return checks
|
2023-01-12 15:48:54 +08:00 |
|
|
|
444d4a69eb
|
i guess this was my first rodeo
|
2023-01-12 15:37:50 +08:00 |
|
|
|
c240f1d2c8
|
Make SavePage API call if we have content provided
|
2023-01-12 10:50:06 +08:00 |
|
|
|
7263e91d6b
|
Enable ts-node files to pull in readability.d.ts
|
2023-01-11 23:59:27 +08:00 |
|
|
|
ee8bad424b
|
Fix typo
|
2023-01-11 23:19:59 +08:00 |
|
|
|
fa866f1d29
|
Make function more clear
|
2023-01-11 23:08:30 +08:00 |
|
|
|
ee50b237c6
|
Fix order of args
|
2023-01-11 22:56:10 +08:00 |
|
|
|
0bc87deba0
|
Change order of ctx arguments
|
2023-01-11 22:55:04 +08:00 |
|
