Benature/omnivore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
468 Commits 1 Branch 0 Tags
be7e36ffeda85bd03d4dc9081aaf416926cd05dd
Commit Graph

95 Commits

Author SHA1 Message Date
Jackson Harper
be7e36ffed Use the validateUrl method to validate URLs 2022-03-07 14:34:32 -08:00
Jackson Harper
ae329d2651 Add test data file 2022-03-03 19:47:37 -08:00
Jackson Harper
b6fd3e786e Fix parsing authors from page metadata 2022-03-03 19:40:02 -08:00
Jackson Harper
49092b707d Remove async 2022-03-03 19:31:51 -08:00
Jackson Harper
05373ba3c7 add methid to parse content metadata 2022-03-03 17:10:06 -08:00
Jackson Harper
c2e08d0e8f Fetch title and author from page metadate if possible 2022-03-03 15:20:58 -08:00
Jackson Harper
b17f22949c Use the readability title if available 2022-03-03 14:21:30 -08:00
Jackson Harper
a4bb67deee Use the readability byline for email authors if available 2022-03-03 14:21:03 -08:00
Jackson Harper
b326a5f8e7 Add more matches on substack icons 2022-03-03 13:42:30 -08:00
Jackson Harper
484cd78ac5 prettier 2022-03-02 23:14:10 -08:00
Jackson Harper
65ce8353dc Attempt to pull URLs for probable newsletter emails out of content 2022-03-02 23:09:10 -08:00
Jackson Harper
21329949e5 Fix to email address in saveNewsletterEmail 2022-03-02 22:06:15 -08:00
Jackson Harper
02d505a7e2 Add some debugging 2022-03-02 21:39:10 -08:00
Jackson Harper
8fe9827ad2 If an email appears to be a newsletter at it to the library 2022-03-02 20:47:53 -08:00
Jackson Harper
65cc666579 Pass HTML instead of a JSDOM into isProbablyNewsletter to better encapsulate 2022-03-02 20:38:11 -08:00
Jackson Harper
c4e237927d Allow any on GCP func 2022-03-02 20:27:40 -08:00
Jackson Harper
e660e41ed5 Fix warnings 2022-03-02 19:49:10 -08:00
Jackson Harper
9206230659 Better name for the save newsletter service 2022-03-02 19:45:28 -08:00
Jackson Harper
d66c114a7d Remove file, this is moved into the parser 2022-03-02 16:36:36 -08:00
Jackson Harper
b5f9478350 Dont mutate function input 2022-03-02 16:34:56 -08:00
Jackson Harper
f7f83fe080 New function to determine if an HTML blob is probably a newsletter based on its content 2022-03-02 16:31:15 -08:00
Jackson Harper
505f888e37 Update to latest intercom client 2022-03-01 11:20:31 -08:00
dependabot[bot]
cadccfc7e1 Bump intercom-client from 2.11.2 to 3.1.4 
Bumps [intercom-client](https://github.com/intercom/intercom-node) from 2.11.2 to 3.1.4.
- [Release notes](https://github.com/intercom/intercom-node/releases)
- [Commits](https://github.com/intercom/intercom-node/commits/v3.1.4)

---
updated-dependencies:
- dependency-name: intercom-client
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 18:40:33 +00:00
Hongbo Wu
fc9aa9452c Add a flag in readability to retain table elements in newsletter emails (#152) 
* add a flag in readability to retain table elements in newsletter emails

* remove header of axios newsletters
 2022-03-01 11:49:38 +08:00
dependabot[bot]
b2faf8a78d Bump @opentelemetry/instrumentation-express from 0.24.0 to 0.28.0 
Bumps [@opentelemetry/instrumentation-express](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.28.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...contrib-test-utils-v0.28.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-express"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-28 21:27:15 +00:00
Hongbo Wu
7bf454ae91 use dataloader to fetch all labels of a list of linkIds in a single q… (#133) 
* use dataloader to fetch all labels of a list of linkIds in a single query and cached

* add labels in GQL query in frontend
 2022-02-28 12:13:26 +08:00
Jackson Harper
328ebc48cb Apply code block highlighting before running DOM clean 2022-02-26 19:04:32 -08:00
Jackson Harper
e0b8cfc938 Import highlightjs types 
We still need these dev packages because of the way our docker
builds are setup.
 2022-02-26 17:37:01 -08:00
Jackson Harper
fd39923907 Prettier improvements 2022-02-26 16:38:43 -08:00
Jackson Harper
84fbc9cd27 Add code highlighting using highlight.js 2022-02-26 14:57:59 -08:00
Satindar Dhillon
e27c2bdd8b generate swift gql code 2022-02-25 10:09:17 -08:00
Hongbo Wu
b9d581166e fix merge conflicts 2022-02-24 14:56:00 +08:00
Hongbo Wu
42f2cffdf8 remove debug log 2022-02-24 14:47:33 +08:00
Hongbo Wu
276205d52a add linkId and labels in article type 2022-02-24 14:44:49 +08:00
Hongbo Wu
c7841b8e8a remove duplicate links because of join with link_labels table 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc18004a5d add labels in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
edb9339897 add labels and link_labels table name 2022-02-24 14:44:49 +08:00
Hongbo Wu
5651c92efe ingore case when checking name uniqueness 2022-02-24 14:44:49 +08:00
Hongbo Wu
9599c98c77 add label filters in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc37d06fd2 update params variable 2022-02-24 14:44:49 +08:00
Hongbo Wu
d33b213092 add setLabels api and tests 2022-02-24 14:44:48 +08:00
Hongbo Wu
9150e7dbc2 add tests for create/delete labels 2022-02-24 14:44:48 +08:00
Hongbo Wu
e74f334db9 update entity 2022-02-24 14:44:48 +08:00
Jackson Harper
6e6745a2b7 Use version 6.1.0 for normalize-url which switches to modules in v7 2022-02-23 10:28:28 -08:00
dependabot[bot]
385cfdad71 Bump normalize-url from 5.3.1 to 7.0.3 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 5.3.1 to 7.0.3.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits/v7.0.3)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-23 18:14:46 +00:00
Jackson Harper
7c9c75836b Update the monitor packages as these are peer dependencies 2022-02-23 09:26:48 -08:00
Jackson Harper
5e2bba3231 Upgrade opentelemetry packages 2022-02-22 12:42:19 -08:00
dependabot[bot]
4a9d24025b Bump @opentelemetry/node from 0.22.0 to 0.24.0 
Bumps [@opentelemetry/node](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/node"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-22 12:42:19 -08:00
Jackson Harper
5f6f3ba3b2 Fix warnings in the API package 2022-02-21 16:46:10 -08:00
Jackson Harper
a276af5264 Remove extra white space 2022-02-21 13:55:09 -08:00