Commit Graph

59 Commits

Author SHA1 Message Date
35ca7ede82 Add test case for danluu 2022-06-06 22:37:05 +08:00
704726dc6a Improve parsing of channel news asia 2022-06-03 14:03:14 -07:00
304fe70113 Fix tests 2022-06-01 10:30:23 +08:00
98ecdcff80 Fix test case of setting tweet-placeholder 2022-05-31 22:51:27 +08:00
404805e0c0 Make async calls to parse() 2022-05-31 22:51:00 +08:00
0b0edd3e69 Make fetching tweet url async 2022-05-31 22:50:14 +08:00
417ed0a4eb Fetch tweet id from url 2022-05-31 20:02:54 +08:00
b6fef171be If we have a node with only one child element which has the placeholder class, keep it 2022-05-31 14:18:39 +08:00
22f5e1cc32 Fix embed tweets got deleted when simplifying nested elements 2022-05-31 13:42:16 +08:00
cc5bdf96f5 Update electrek test case 2022-05-31 12:52:33 +08:00
bce50c2a92 Update electrek test case 2022-05-31 11:11:06 +08:00
9dc7fd4c4c Add test case for electrek 2022-05-30 22:47:47 +08:00
a34806a782 Fix tests 2022-05-26 10:55:21 +08:00
7d4d1d7b67 Parse language in readability 2022-05-26 10:55:21 +08:00
35dcd00ec3 Add test case for github blog 2022-05-18 15:52:47 +08:00
0e4cec5e25 Update tests 2022-05-18 11:28:47 +08:00
6795508942 Add test for readability on youtube embeds 2022-05-18 09:55:36 +08:00
bdfa76d716 Remove listnav elements from articles 2022-05-12 15:58:24 -07:00
d542d31aed Fix gflownet test generation 2022-05-11 21:23:30 +08:00
2755da16a9 Fix not getting iframe src 2022-05-11 19:25:12 +08:00
82fb8151a4 Fix generate tests 2022-05-10 21:10:20 +08:00
96b543946d Temporarily disable customer content serializer test 2022-05-10 21:10:06 +08:00
0eb565eae9 Remove JSDOM dependencies from readability 2022-05-10 21:09:19 +08:00
ffa5dee721 Use linkedom in readability tests 2022-05-10 18:40:52 +08:00
cb7f30607a Use linkedom in readability test isProbablyReaderable 2022-05-10 18:33:29 +08:00
7d8379d093 Use linkedom in readability benchmark 2022-05-10 18:32:55 +08:00
39d4745591 Move linkedom dependency in devDependencies in readability 2022-05-10 18:31:58 +08:00
2152a9e466 Fix getting embeded class lists bug 2022-05-10 16:57:38 +08:00
76d47f7dc5 Fix updating live collections 2022-05-10 16:57:02 +08:00
79a941a2b6 Default use options.url if exists 2022-05-10 16:56:09 +08:00
cc48034145 Add linkedom to dependencies 2022-05-10 16:54:14 +08:00
eaad96acdd Return parsed dom back to backend 2022-05-06 12:29:08 +08:00
5f5076e864 Highlight code element without reinitialize jsdom 2022-05-06 12:20:54 +08:00
6ef14e1f91 Mark ft.com page a readerable 2022-05-05 09:28:22 -07:00
384c5dbf9f Improve rendering of the Financial Times 2022-05-05 09:14:51 -07:00
24373018af Return non-text elements if no text content found after parsing in readability 2022-05-05 19:26:39 +08:00
74693d40c0 Mark page as readerable 2022-05-04 15:01:35 -07:00
8386aebaf8 Remove ads from fiercepharma page 2022-05-04 14:22:08 -07:00
a24b976546 Remove lazy loaded srcset elements
Some tools like jetpack: https://jetpack.com/support/lazy-images/
use a temporary srcset element set to a data image when lazy
loading, these are later removed by JS. We test if there is
a valid src attribute and if the srcset attribute is a data embed
to remove these.
2022-04-29 10:05:33 -07:00
d2bb359f5c Handle srcset image density specifications 2022-04-26 15:18:23 -07:00
72a231c97e Do not proxy image data uri (#421)
* do not proxy image data uri

* rename data uri

* rename data uri in test
2022-04-14 12:59:12 +08:00
4d01f689b2 replace tables of article content with divs for newsletters 2022-04-11 20:00:11 +08:00
da28998130 Pull ul list out of newsletter blurb
next/react doesnt want child elements of the paragraphs

Improve formatting

Improve wording

Use buttons in the subscribe directly blocks

Simplify docs on setting up forwarding rules

Add extra padding on bottom of help docs

Remove unused style

Add emails help page

Improve formatting

Prefetch page content on iOS

Reduce the reader overly length now that items are precached

Add invalidation when highlights are added to items

fix missing index_settings.json file in api dockerfile for creating elastic index (#363)

Handle full email address objects in the to param from sendgrid

These come in a format like:

"jacksonh-dfdf@inbox.omnivore.app" <jacksonh-dfdf@inbox-demo.omnivore.app>

New IDs for short highlights because they dont cascade delete now

Testing CI issues

Simplify test

CI test

Use promises for async tests

Temporarily remove test to debug CI

Re-enable

re-enable test, return error

Specify a userId when looking up saved email pages

create a unique url for newsletters without a URL

Use 500ms on page test timeouts

Increase timeout

Dont use deep equal to match newsletter label

Run just the labels API

Run against just the newsletter emails

Run without the page tests

Fix

Set the allow uncaught flag

Remove highlight tests

Remove newsletters tests

more resolver tests

Remove newsetter tests

Comment out resolver tests

Use nock for external requests in tests

Specify puppeteer url for tests

Comment out more tests

uncomment tests

re-enable

re-enable email test

Re-disable

Re-enable one pdf attachment test

Re-disable pdf attachment test

Use promises on setTimeout tests

rm label tests

mv label tests into a context

Comment out pdf tests

Comment out pdf tests

Async test

Async wrappers

Delay when creating test pages

More debugging

Unique short ids

Remove potentially problematic test

Fetch page before returning for test

handler in before block

more debugging

More debugging

Move errors checks into contexts

Use a context when saving newsletters to force index refresh

Prettier fix

Fix newsletter label check, remove setTimeout

Re-enable test

timeout on pdf router handler

Fix method call

comment out PDF test

Unique fake username

Comment out PDF test

Debugging signed urls

Re-enable

New email

pdf test

PDF tests

Comment out pdf test

Add nock stubs for email URLs

Use full address for PDF test

Remove debug

Use full email addresses
2022-04-02 16:56:24 -07:00
6d405432af add site_name and site_icon to page model and return in resolver (#341)
* add site_name and site_icon to page model and return in resolver

* fix tests
2022-03-30 10:43:10 +08:00
5036924b21 Merge pull request #280 from omnivore-app/dependabot/npm_and_yarn/jsdom-19.0.0
Bump jsdom from 16.7.0 to 19.0.0
2022-03-25 09:08:55 -07:00
12af64609c Fix readability issues with null style elements
isProbably visible can fail in this case because style could be
undefined on an element.
2022-03-23 13:35:00 -07:00
3ec6f22199 Bump jsdom from 16.7.0 to 19.0.0
Bumps [jsdom](https://github.com/jsdom/jsdom) from 16.7.0 to 19.0.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/master/Changelog.md)
- [Commits](https://github.com/jsdom/jsdom/compare/16.7.0...19.0.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-23 18:51:08 +00:00
960a22d50c Fix/city journal parsing (#266)
* remove arrow image when parsing

* ignore m_article classname element which indicates a mobile version of the website

* generate test page for city journal
2022-03-21 22:53:21 +08:00
c79e740ceb Add package 2022-03-14 11:13:59 -07:00
0361ef86fa Better handling of HTML entities in descriptions
The HTML code method didnt implent all possible
entities, causing some (usually rquote) to display.
2022-03-14 11:02:08 -07:00