ba8b6796f3
Merge pull request #1746 from omnivore-app/fix/save-youtube-playlist
...
Fix incorrect Youtube playlist author name and description
2023-01-31 11:06:25 +08:00
61bf89b9ad
Add tests
2023-01-30 19:10:27 +08:00
4f12580b1c
Use email header x-beehiiv-type to detect beehiiv newsletter and get the newsletter url from the header x-newsletter
2023-01-30 19:03:15 +08:00
88f6e20bfc
Pass email headers to the content-handler
2023-01-30 18:52:38 +08:00
ec7344b923
Get playlist id from the URL and retrieve the information of the playlist
2023-01-30 16:53:19 +08:00
1b3d4bbffa
Do not escape the title of youtube video or tweet which is saved in elastic
2023-01-18 19:08:36 +08:00
6133d0865a
Set title for YouTube based on oembed data
2023-01-16 20:03:23 +08:00
145a8b9730
Fix last two tweets not saved in a long thread by waiting for document loaded after scrolling
2023-01-12 22:03:29 +08:00
1243e2ace2
Fix the error when no name in subscription by using the email address as the author when no author found in the newsletter
2023-01-10 11:17:22 +08:00
ab16447c14
Fix not saving all the tweets in a thread
2023-01-03 21:26:34 +08:00
3a3a06b5d6
Fix vote count
2022-11-29 16:18:09 +08:00
1fe86b6230
Parse owner
2022-11-29 15:48:45 +08:00
f8e0efad05
fix bug
2022-11-29 15:40:22 +08:00
8ee19a98fa
parse authors
2022-11-29 12:26:37 +08:00
c993fa11be
Add stack-overflow content-handler
2022-11-29 11:39:23 +08:00
db4f225b5c
Merge pull request #1426 from omnivore-app/fix/github-handler-titles
...
Use greedy matching on the GitHub title regex
2022-11-17 13:42:23 +08:00
c18217e697
Fix logRecord being shared by requests
2022-11-17 11:28:46 +08:00
d9237723c5
Share puppeteer browser instance between content handlers
2022-11-17 11:28:46 +08:00
e866541ae1
Fix puppeteer launch in head mode
2022-11-17 11:28:46 +08:00
8bdbd86e97
Make headless = false in local testing
2022-11-17 11:27:51 +08:00
0d83ecb068
Use greedy matching on the GitHub title regex
...
This fixes issues with titles that have slashes (/) in their title
as the regex match would look for the last possible matching
slash.
2022-11-16 13:52:58 +08:00
c8febe900f
Fix prettier error
2022-11-16 10:34:52 +08:00
726f69484f
Better content extraction of title and description for GitHub URLs
2022-11-15 18:43:55 +08:00
9d753e3c0e
Forward convertkit confirmation email
2022-11-02 15:27:30 +08:00
ae3ff6f34b
Add test case
2022-11-01 17:48:39 +08:00
67082a478e
Add support for hey world newsletters
2022-11-01 16:41:42 +08:00
8712a3efef
Merge pull request #1373 from omnivore-app/fix/substack-embedded-tweets
...
Handle embedded tweets in substack emails
2022-11-01 15:04:03 +08:00
e1d7eeed01
Make test verify preparse is true
2022-11-01 10:46:31 +08:00
9b29a3c261
querySelectorAll returns a NodeList which we can iterate
2022-11-01 10:34:18 +08:00
392ac15a62
Treat all tweets as elements so we dont need to cast
2022-11-01 10:31:12 +08:00
a4f0698974
Handle older style substack image embeds as well as new
2022-10-31 21:57:29 +08:00
734c4337df
Add missing test file
2022-10-31 21:51:22 +08:00
2622437831
Fix linting
2022-10-31 21:42:31 +08:00
cc91e43572
Handle embedded tweets in substack emails
...
This does a few things:
- tags static tweets found in substack emails with a special class
- upgrades readability to ignore special class names
- reduces some readability debug output
2022-10-31 21:28:36 +08:00
a7b208fd26
Skip non-author replies in twitter thread
2022-10-31 16:51:20 +08:00
1067747cc2
Add twitter author when saved
2022-10-27 19:02:16 +08:00
a536ed9b5e
Save newer thread with puppeteer too
2022-10-27 14:20:20 +08:00
b556bc17e8
Set navigation timeout to be 60 seconds
2022-10-27 13:05:35 +08:00
6ca1e7dc25
Add no-sandbox in the puppeteer args
2022-10-27 12:49:00 +08:00
03777587f2
Fetch max 100 tweets in a thread
2022-10-26 22:44:19 +08:00
b5926ccf1c
Get old tweet thread with puppeteer and new tweet with twitter api
2022-10-26 20:41:51 +08:00
5f0d0ed69b
Get tweet reply ids from thread id using puppeteer
2022-10-25 22:09:16 +08:00
343c8fb5cc
Get tweet reply ids from thread id
2022-10-25 18:25:49 +08:00
31bc9fe5be
Fix node-weekly newsletter getting forwarded
...
* This error is caused by a previous rebase conflict and resolving the conflict caused an important line missing
* Revert the change to add CooperPressHandler to the array of newsletter handlers
* Updated the test to get the correct handler from email content to prevent such cases
2022-10-21 09:57:05 +08:00
57846a1c5e
Fix tests
2022-10-18 15:14:38 +08:00
706607ea2b
Rebase main
2022-10-18 14:54:33 +08:00
1fda6946bb
Save newsletters hosted by cooper-press.com correctly
2022-10-18 14:53:02 +08:00
9757394a05
Merge pull request #1318 from omnivore-app/fix/newsletter-ghost-org
...
Save newsletters hosted by ghost.org correctly
2022-10-18 14:50:27 +08:00
59427fa896
Mock the substack domain also when testing redirects
...
We were making the actual call to the redirected URL here,
instead of mocking it. We need to mock both the embedded email
domain, and the domain we are redirected to.
2022-10-18 12:51:53 +08:00
d8c5c97960
Save newsletters hosted by ghost.org correctly
2022-10-17 19:27:02 +08:00
