|
|
b746046727
|
update default score
|
2023-06-23 12:34:31 +08:00 |
|
|
|
9ef40b3ed9
|
deduct latency from the score of the member in sorted set
|
2023-06-23 12:27:34 +08:00 |
|
|
|
ae504a970b
|
save instances in redis as a sorted set
|
2023-06-23 12:17:41 +08:00 |
|
|
|
4607754f58
|
reduce timeout to 20s
|
2023-06-23 11:36:30 +08:00 |
|
|
|
6b203dde4c
|
update instances pool to pick one from each country
|
2023-06-23 10:54:43 +08:00 |
|
|
|
61542a34bd
|
fix: create an instance pool to scrape nitter
|
2023-06-23 10:45:37 +08:00 |
|
|
|
30b7c38e76
|
fix: increase twitter-handler timeout to 60 seconds
|
2023-06-22 21:16:55 +08:00 |
|
|
|
12009d1d06
|
fix: tweet author not saved
|
2023-06-22 18:05:53 +08:00 |
|
|
|
0a0d64ca4e
|
Merge pull request #2409 from omnivore-app/fix/nitter-scraper
using axios to fetch html from nitter.net
|
2023-06-22 15:01:28 +08:00 |
|
|
|
3e2e16e915
|
skip running readability on scraped tweet
|
2023-06-22 15:00:54 +08:00 |
|
|
|
0d7adcdecc
|
using axios to fetch html from nitter.net
|
2023-06-22 14:32:18 +08:00 |
|
|
|
a0556055c5
|
Merge pull request #2389 from Podginator/feat/atlantic-handler
Add The Atlantic handler to avoid paywall and correctly format
|
2023-06-22 10:21:13 +08:00 |
|
|
|
de2ca58b37
|
Linting fix
|
2023-06-22 09:52:43 +08:00 |
|
|
|
6aceadc819
|
fix attachment url
|
2023-06-21 17:46:05 +08:00 |
|
|
|
f0e6458e4a
|
filter out replies
|
2023-06-21 16:01:09 +08:00 |
|
|
|
a3ad9375bf
|
replace url in tweets
|
2023-06-21 15:50:55 +08:00 |
|
|
|
d2094981ad
|
increase timeout value to 60 seconds
|
2023-06-21 15:21:40 +08:00 |
|
|
|
dde104b95e
|
fix video attachment not saved
|
2023-06-21 15:07:21 +08:00 |
|
|
|
a109d0383f
|
go to the next thread page to scrape long thread
|
2023-06-20 19:17:34 +08:00 |
|
|
|
89f89875d6
|
fix date parsing
|
2023-06-20 17:35:59 +08:00 |
|
|
|
7293078068
|
Remove Audio section for Feature Articles
|
2023-06-20 10:35:04 +02:00 |
|
|
|
44dd44dd58
|
update comments on tests
|
2023-06-20 10:26:53 +02:00 |
|
|
|
c0f34270ee
|
Remove Related content links from article (IE: Read article X)
|
2023-06-20 10:24:41 +02:00 |
|
|
|
55fd02b67a
|
Fix Tests
|
2023-06-20 09:20:43 +02:00 |
|
|
|
ff9fcef08d
|
disable twitter-handler
|
2023-06-20 14:59:14 +08:00 |
|
|
|
65758f3469
|
feat: add nitter handler to scrape tweets from twitter.com and nitter.net
|
2023-06-20 14:56:54 +08:00 |
|
|
|
8e13284795
|
Linting fixes
|
2023-06-20 13:48:18 +08:00 |
|
|
|
0cae728a43
|
Add missing files
|
2023-06-20 02:42:09 +02:00 |
|
|
|
a08af0610a
|
Add The Atlantic handler to avoid paywall and correctly format
|
2023-06-20 02:38:24 +02:00 |
|
|
|
fb2c70a6fd
|
change the page viewport when scraping twitter thread
|
2023-06-15 18:33:26 +08:00 |
|
|
|
d69e732579
|
fix: dedupe url if failed to save
|
2023-06-01 16:11:35 +08:00 |
|
|
|
8ddde61285
|
fix: increase twitter-handler timeout to 60 seconds
|
2023-06-01 16:11:35 +08:00 |
|
|
|
4d422402f8
|
fix: convert published time to GMT +8 in the parsed wechat blogs
|
2023-05-24 10:55:54 +08:00 |
|
|
|
94aead9988
|
Add content handler for weixin.qq.com
|
2023-05-16 14:46:44 +08:00 |
|
|
|
89c85e5627
|
Merge pull request #2109 from omnivore-app/feat/tweet-titles
Better tweet titles
|
2023-04-25 14:17:33 +08:00 |
|
|
|
6560a6e694
|
Add type definitions
|
2023-04-25 12:23:37 +08:00 |
|
|
|
155fad5b9c
|
Truncate tweet text used in titles using lodash
|
2023-04-25 11:39:55 +08:00 |
|
|
|
53e81cc06e
|
Better formatting of tweet titles
|
2023-04-25 09:47:52 +08:00 |
|
|
|
187ad654e7
|
Add debug logs
|
2023-04-22 12:29:46 +08:00 |
|
|
|
2760ce60f0
|
Fix list-unsubscribe header undefined
|
2023-04-22 11:07:15 +08:00 |
|
|
|
89ac9c46cc
|
Remove other places that use rfc2047 decoder
|
2023-04-20 20:53:06 +08:00 |
|
|
|
06733227d6
|
Remove new "restack" and "share" buttons from SubStack newsletters
|
2023-04-18 14:53:27 +08:00 |
|
|
|
6c27c5385f
|
Remove footnote links in wikipedia articles
|
2023-04-13 12:10:07 +08:00 |
|
|
|
e03301e294
|
Merge pull request #2026 from omnivore-app/fix/embedded-quote-tweets
Handle embedded quote tweets
|
2023-04-12 16:23:38 +08:00 |
|
|
|
c366cd6472
|
Fix test value, there is only one quote tweet
|
2023-04-12 15:37:26 +08:00 |
|
|
|
ae45067a32
|
Add test data file
|
2023-04-12 15:16:11 +08:00 |
|
|
|
344514bd0f
|
Handle embedded quote tweets
|
2023-04-12 14:50:20 +08:00 |
|
|
|
4969391d4c
|
Replace picture with image
|
2023-04-12 12:15:53 +08:00 |
|
|
|
eb58bf11ba
|
Force to use content handler of piped.video when saving from extensions
|
2023-04-10 20:52:09 +08:00 |
|
|
|
bfbd80e8f1
|
Get the published date
|
2023-04-06 16:41:21 +08:00 |
|
