Benature/omnivore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,658 Commits 1 Branch 0 Tags
60bbbb6cf3e5993bed3e1e6bc956e80fc9ed6a32
Commit Graph

268 Commits

Author SHA1 Message Date
Hongbo Wu
e1e0ddf7fc Merge pull request #582 from omnivore-app/optimize-parsing 
Optimize parsing
 2022-05-12 11:07:52 +08:00
Hongbo Wu
602d141dec Rename doc to dom 2022-05-12 11:00:32 +08:00
Hongbo Wu
5dca02873d Revert originalArticleUrl change 2022-05-12 10:45:13 +08:00
Hongbo Wu
8f75c43658 Add siteName in search result 2022-05-12 10:30:47 +08:00
Hongbo Wu
76aef35423 Remove www. from the siteName in the search api response 2022-05-11 23:07:34 +08:00
Hongbo Wu
0b11c31317 Add linkedom dependency in packages/api 2022-05-10 18:31:25 +08:00
Hongbo Wu
93ed1936f0 Remove jsdom in packages/api 2022-05-10 17:12:52 +08:00
Hongbo Wu
a78a6c6ba4 Replace DomWindow with Document in handlers 2022-05-10 17:01:23 +08:00
Hongbo Wu
6a57281e74 Remove DomWindow usage 2022-05-10 17:00:56 +08:00
Hongbo Wu
acc7654a2f Replace jsdom with linkedom 2022-05-10 16:59:09 +08:00
Hongbo Wu
5698790288 Pass url to readability 2022-05-10 16:53:45 +08:00
Hongbo Wu
5007b11a3c Add url to readability option 2022-05-10 16:52:10 +08:00
Hongbo Wu
59a2639b7d Reduce http call to get jsonld data if title or content or sitename or byline exists 2022-05-09 13:45:45 +08:00
Hongbo Wu
a457c9d128 Update article content only when code blocks exist 2022-05-09 13:45:45 +08:00
Hongbo Wu
eaad96acdd Return parsed dom back to backend 2022-05-06 12:29:08 +08:00
Hongbo Wu
5f5076e864 Highlight code element without reinitialize jsdom 2022-05-06 12:20:54 +08:00
Hongbo Wu
7c6b810522 Remove redundant JSDOM 2022-05-06 10:53:36 +08:00
Hongbo Wu
21ff9a5ae7 Add test for state=failed or processing 2022-05-04 21:42:59 +08:00
Hongbo Wu
93ab6bc87a Add includePending=true in search API 2022-05-04 21:28:06 +08:00
Hongbo Wu
bc1ed3f054 Check state in articleSavingRequest 2022-05-04 19:33:13 +08:00
Hongbo Wu
9e3db0e053 Fix a bug to have multiple pages with the same url in lib 2022-05-04 19:12:22 +08:00
Hongbo Wu
afe8b6e948 make savedAt a required field in page 2022-05-04 17:54:59 +08:00
Hongbo Wu
a42bb7ab38 Replace createdAt with savedAt 2022-05-04 12:59:07 +08:00
Hongbo Wu
98ea1016d5 Remove <p> tag in description 2022-05-04 12:50:22 +08:00
Hongbo Wu
9048ccbab3 Show failed pages in library 2022-05-04 12:46:57 +08:00
Hongbo Wu
ca0f58ee22 Update page content and state if failed to parse 2022-05-04 12:21:41 +08:00
Hongbo Wu
80d0146f72 Show unable to parse if page is failed to save within 30 seconds 2022-05-04 11:55:16 +08:00
Jackson Harper
c8c4377346 Merge pull request #531 from omnivore-app/fix/saving-analytics 
Add more detailed analytics for saving. Remove intercom events
 2022-05-02 21:14:18 -07:00
Jackson Harper
95b163a437 Prettier clean up 2022-05-02 20:58:05 -07:00
Jackson Harper
ab96bb09ab Add more detailed analytics for saving. Remove intercom events 2022-05-02 20:41:10 -07:00
Jackson Harper
b649d583b7 Linting changes 2022-05-02 15:58:40 -07:00
Jackson Harper
453b301e74 Allow looking up articles by ID 
iOS needs to be able to lookup an article by its ID not its
slug, because the slug can change once saved.
 2022-05-02 15:48:21 -07:00
Jackson Harper
9e94ed5067 We need node-gyp when building API in docker 2022-05-02 11:00:39 -07:00
Jackson Harper
61b7ac3cb2 Enable graphql introspection when running locally 2022-05-02 09:44:13 -07:00
Hongbo Wu
77ca0ceb59 Show empty description if not exist 2022-04-29 15:50:39 +08:00
Hongbo Wu
2b70d480d2 Remove article saving request (#493) 
* Add state and taskName in elastic page mappings

* Add state and taskName in elastic page interface

* Create page with PROCESSING state before scrapping

* Update createArticleRequest API

* Fix tests

* Add default state for pages

* Update createArticle API

* Update save page

* Update save file

* Update saving item description

* Show unable to parse content for failed page

* Fix date parsing

* Search for not failed pages

* Fix tests

* Add test for saveUrl

* Update get article saving request api

* Update get article test

* Add test for articleSavingRequest API

* Add test for failure

* Return new page id if clientRequestId empty

* Update clientRequestId in savePage

* Update clientRequestId in saveFile

* Replace article with slug in articleSavingRequest

* Add slug in articleSavingRequest response

* Depreciate article

* Use slug in web

* Remove article and highlight fragments

* Query article.slug on Prod

* Show unable to parse description for failed page

* Fix a bug having duplicate pages when saving the same url multiple times

* Add state in response

* Rename variables in removeArticle API

* Rename state

* Add state in response in web

* Make state an enum

* Open temporary page by link id

* Use an empty reader view as the background for loading pages

* Progressively load the article page as content is loaded

* Add includePending flag in getArticles API

* Set includePending = true in web

* Add elastic update mappings in migration script

* Add elastic mappings in docker image

* Move index_settings.json to migrate package

* Remove elastic index creation in api

* Move elastic migrations to a separate directory

* Remove index_settings from api docker image

Co-authored-by: Jackson Harper <jacksonh@gmail.com>
 2022-04-29 13:41:06 +08:00
Hongbo Wu
ccb79845bc Fix subscriptions issues (#501) 
* Replace fetch with axios

* Add url in subscription

* Remove url in subscription

* Unsubscribe all newsletters before deleting that email

* Continue unsubscribing others if one got error
 2022-04-29 11:16:46 +08:00
Hongbo Wu
d1c2721b95 Add expiration date to api key generation (#495) 
* Add exp to jwt token

* Add expiredAt in generate api key input

* Check exp in claim

* Add tests
 2022-04-28 09:09:40 +08:00
Jackson Harper
9362aa54c7 Merge pull request #443 from omnivore-app/dependabot/npm_and_yarn/cookie-0.5.0 
Bump cookie from 0.4.2 to 0.5.0
 2022-04-22 13:07:59 -07:00
Hongbo Wu
bcbda50daa Add basic UI for subscriptions 2022-04-22 12:17:09 -07:00
Hongbo Wu
3e35fb1aca Email content display report 2022-04-22 19:14:59 +08:00
Hongbo Wu
71aa66ea9c Allow search for subscription:name (#461) 2022-04-22 08:47:41 +08:00
Jackson Harper
0a4d55ef15 Increase the rate limit 
Turns out 10 is very low because we make CORs requests from the
browser and SWR will make background requests for some objects
like viewer. I was hitting the limit while testing search.
 2022-04-21 14:59:57 -07:00
Hongbo Wu
dca6461b62 Add unsubscribe resolver (#453) 
* add unsubscribe resolver

* add subscription to articles api response

* revert yarn.lock changes
 2022-04-21 13:18:15 +08:00
dependabot[bot]
1b27bf1f4b Bump cookie from 0.4.2 to 0.5.0 
Bumps [cookie](https://github.com/jshttp/cookie) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/jshttp/cookie/releases)
- [Changelog](https://github.com/jshttp/cookie/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/cookie/compare/v0.4.2...v0.5.0)

---
updated-dependencies:
- dependency-name: cookie
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-21 01:43:26 +00:00
Hongbo Wu
83cdec42f7 Add rate limiter for api request (#454) 
* Add rate limiter for api request

* apply rate limit on Demo/Prod
 2022-04-21 09:41:33 +08:00
Hongbo Wu
5c7f4a22bf Save subscriptions when saving newsletters (#447) 
* parse unsub mailTo or httpUrl and send to pubsub

* save subscription from newsletter

* fix tests
 2022-04-20 09:24:40 +08:00
Jackson Harper
48e36c8635 Merge pull request #393 from omnivore-app/OMN-382 
[Omn-382] - Updated landing page
 2022-04-19 11:15:31 -07:00
Hongbo Wu
d7a2659fc4 fix searching for multiple labels (#444) 2022-04-19 12:13:47 +08:00
Hongbo Wu
1117a0c575 Feature/subscription list resolver (#432) 
* add subscriptions table

* add listSubscriptions schema

* add listSubscriptions resolver
 2022-04-19 11:08:43 +08:00