|
|
a03eee5ef7
|
fix dependecies
|
2024-01-18 18:48:46 +08:00 |
|
|
|
d9feb740cb
|
convert content-fetch to typescript
|
2024-01-18 18:48:46 +08:00 |
|
|
|
cd3402b98a
|
rewrite puppeteer in typescript
|
2024-01-18 18:48:46 +08:00 |
|
|
|
51e586ed3d
|
separate content-fetch in puppeteer packages from saving page content
|
2024-01-18 18:48:46 +08:00 |
|
|
|
ad63c75e63
|
fix typo
|
2023-12-08 11:29:03 +08:00 |
|
|
|
3759e10615
|
fix feed url in pdf file not saved
|
2023-12-08 11:29:02 +08:00 |
|
|
|
d09ec51136
|
Merge pull request #3182 from omnivore-app/fix/importer-notification
|
2023-11-28 14:59:52 +08:00 |
|
|
|
b10b704da3
|
fix importer metrics not updated when failed to catch invalid url in the list
|
2023-11-28 12:14:27 +08:00 |
|
|
|
fd781644f1
|
feat: fetch content for rss feed items in following folder
|
2023-11-23 18:03:25 +08:00 |
|
|
|
c4773dc904
|
Landing page improvements and various supporting improvements
|
2023-10-24 09:43:39 +01:00 |
|
|
|
1b1cce7485
|
disable javascript for the host
|
2023-10-20 18:59:22 +08:00 |
|
|
|
d746510358
|
cont
|
2023-10-19 21:50:16 +08:00 |
|
|
|
f750648824
|
fix importer triggers thumbnailer unexpectedly
|
2023-10-19 21:46:43 +08:00 |
|
|
|
0fcc7096aa
|
docs: fix typo in packages/puppeteer-parse/README.md
|
2023-10-18 17:33:22 +05:45 |
|
|
|
00bd183287
|
do not retry importer job if user account is deleted
|
2023-10-16 16:33:22 +08:00 |
|
|
|
5f6be169bd
|
add savedAt and publishedAt to saveUrl api
|
2023-08-14 17:10:34 +08:00 |
|
|
|
792cf0b207
|
upgrade google cloud sdk
|
2023-08-11 13:55:29 +08:00 |
|
|
|
7641a2567e
|
disable extensions too
|
2023-08-02 16:12:24 +08:00 |
|
|
|
4eab6ea6d2
|
remove hardware acceleration
|
2023-08-02 16:07:43 +08:00 |
|
|
|
a97fcd1e88
|
do not use single process in chromium
|
2023-08-02 15:58:32 +08:00 |
|
|
|
63cbb3011e
|
upgrade puppeteer and update chromium args
|
2023-08-02 15:33:15 +08:00 |
|
|
|
153662ce6a
|
set savedAt and publishedAt timestamp for rss item
|
2023-07-14 12:21:34 +08:00 |
|
|
|
0170ea3109
|
fetch rss item by url
|
2023-07-13 10:09:48 +08:00 |
|
|
|
fd67baea09
|
remove the code to get create_time from wechat article in browser runtime
|
2023-07-11 16:20:08 +08:00 |
|
|
|
1f283e6122
|
add optional locale and timezone to saveUrl request for puppeteer to use
|
2023-07-11 16:15:32 +08:00 |
|
|
|
244fb4ccb5
|
fix: removing node with background image
|
2023-06-26 16:40:14 +08:00 |
|
|
|
43dcb8d5d7
|
fix: invalid url in puppeteer
|
2023-06-26 15:08:46 +08:00 |
|
|
|
d2e476a3ea
|
add debug logs
|
2023-06-14 12:53:43 +08:00 |
|
|
|
9fbe32b219
|
mark import failed on the last failed retry
|
2023-06-13 17:30:26 +08:00 |
|
|
|
4a438f45f5
|
add timeout and content length to the axios requests
|
2023-06-12 21:31:35 +08:00 |
|
|
|
a4449e6f93
|
better handling error in puppeteer
|
2023-06-12 21:05:54 +08:00 |
|
|
|
345c56c57a
|
fix: catch importer status update error
|
2023-06-01 16:11:35 +08:00 |
|
|
|
813a90e3c9
|
feat: send importer status update
|
2023-06-01 16:11:35 +08:00 |
|
|
|
67014e68fd
|
feat: send email notification once the import has finished
|
2023-06-01 16:11:35 +08:00 |
|
|
|
19facec4e2
|
feat: add importer metrics collector
|
2023-06-01 16:11:35 +08:00 |
|
|
|
d69e732579
|
fix: dedupe url if failed to save
|
2023-06-01 16:11:35 +08:00 |
|
|
|
8ddde61285
|
fix: increase twitter-handler timeout to 60 seconds
|
2023-06-01 16:11:35 +08:00 |
|
|
|
df7e1fd269
|
fix: catch scrapingbee api error
|
2023-06-01 16:11:35 +08:00 |
|
|
|
014dc773e3
|
Fix labels not saved correctly by saveUrl
|
2023-04-12 17:04:59 +08:00 |
|
|
|
5c46903fbe
|
Add labels and state to puppeteer-parse
|
2023-04-12 16:50:11 +08:00 |
|
|
|
9c96a4eab2
|
fix puppeteer throwing error when create_time is not defined
|
2023-03-15 13:15:21 +08:00 |
|
|
|
aae6759bcb
|
return published date if the class name is omnivore-published-date which we added when we scraped the article
|
2023-03-13 12:08:01 +08:00 |
|
|
|
1002e0baeb
|
improve regex for url parse
|
2023-03-07 13:51:46 +08:00 |
|
|
|
e39a7cc16b
|
improve urlParse for puppeteer-parse
|
2023-03-07 12:13:00 +08:00 |
|
|
|
69b87078a5
|
Fix not timeout if scroll more than 5 seconds
|
2023-02-13 22:10:56 +08:00 |
|
|
|
d816ee9563
|
Allow image and media to be loaded in puppeteer
|
2023-02-13 16:50:39 +08:00 |
|
|
|
145a8b9730
|
Fix last two tweets not saved in a long thread by waiting for document loaded after scrolling
|
2023-01-12 22:03:29 +08:00 |
|
|
|
7bfd36e308
|
Fix readability not run in puppeteer-parse
|
2023-01-12 18:33:47 +08:00 |
|
|
|
611a50e967
|
Merge pull request #1559 from omnivore-app/feat/importer
New upload file importer
|
2023-01-04 15:32:36 +08:00 |
|
|
|
ab16447c14
|
Fix not saving all the tweets in a thread
|
2023-01-03 21:26:34 +08:00 |
|
