Commit Graph

175 Commits

Author SHA1 Message Date
46c5b8c82a Another attempt at slimming docker file 2022-04-04 10:42:04 -07:00
946734d76c Revert docker changes
These were creating fatter images than we want because of the
lerna dependencies.
2022-04-04 09:59:18 -07:00
bd6bed746e Remove clean command 2022-04-04 09:19:25 -07:00
b15a7b0bd5 Only install dev deps on the prod image 2022-04-04 09:15:56 -07:00
9845a49040 Remove semicolon 2022-04-04 08:17:46 -07:00
b8b471d369 Move @types into dev 2022-04-04 08:17:00 -07:00
c8ec7d636e Use require to import address parser function 2022-04-04 08:11:26 -07:00
87077be85e First stage of build is development 2022-04-04 08:11:06 -07:00
b668114c33 Fix multistage docker build 2022-04-04 07:50:16 -07:00
fc17daa546 Remove unneeded mocha params 2022-04-02 17:02:11 -07:00
f6cfa2f9d2 Revert test files that dont need to be changed 2022-04-02 17:00:45 -07:00
da28998130 Pull ul list out of newsletter blurb
next/react doesnt want child elements of the paragraphs

Improve formatting

Improve wording

Use buttons in the subscribe directly blocks

Simplify docs on setting up forwarding rules

Add extra padding on bottom of help docs

Remove unused style

Add emails help page

Improve formatting

Prefetch page content on iOS

Reduce the reader overly length now that items are precached

Add invalidation when highlights are added to items

fix missing index_settings.json file in api dockerfile for creating elastic index (#363)

Handle full email address objects in the to param from sendgrid

These come in a format like:

"jacksonh-dfdf@inbox.omnivore.app" <jacksonh-dfdf@inbox-demo.omnivore.app>

New IDs for short highlights because they dont cascade delete now

Testing CI issues

Simplify test

CI test

Use promises for async tests

Temporarily remove test to debug CI

Re-enable

re-enable test, return error

Specify a userId when looking up saved email pages

create a unique url for newsletters without a URL

Use 500ms on page test timeouts

Increase timeout

Dont use deep equal to match newsletter label

Run just the labels API

Run against just the newsletter emails

Run without the page tests

Fix

Set the allow uncaught flag

Remove highlight tests

Remove newsletters tests

more resolver tests

Remove newsetter tests

Comment out resolver tests

Use nock for external requests in tests

Specify puppeteer url for tests

Comment out more tests

uncomment tests

re-enable

re-enable email test

Re-disable

Re-enable one pdf attachment test

Re-disable pdf attachment test

Use promises on setTimeout tests

rm label tests

mv label tests into a context

Comment out pdf tests

Comment out pdf tests

Async test

Async wrappers

Delay when creating test pages

More debugging

Unique short ids

Remove potentially problematic test

Fetch page before returning for test

handler in before block

more debugging

More debugging

Move errors checks into contexts

Use a context when saving newsletters to force index refresh

Prettier fix

Fix newsletter label check, remove setTimeout

Re-enable test

timeout on pdf router handler

Fix method call

comment out PDF test

Unique fake username

Comment out PDF test

Debugging signed urls

Re-enable

New email

pdf test

PDF tests

Comment out pdf test

Add nock stubs for email URLs

Use full address for PDF test

Remove debug

Use full email addresses
2022-04-02 16:56:24 -07:00
250118de45 fix missing index_settings.json file in api dockerfile for creating elastic index (#363) 2022-04-01 13:07:44 +08:00
6d405432af add site_name and site_icon to page model and return in resolver (#341)
* add site_name and site_icon to page model and return in resolver

* fix tests
2022-03-30 10:43:10 +08:00
2bada0b9df Merge pull request #293 from omnivore-app/feature/priority-by-rate-limit
Use lower priority queue for large imports
2022-03-28 13:19:35 -07:00
d9af397b9c Merge pull request #331 from omnivore-app/fix/update-page-after-deletion
fix document_missing_exception being thrown to sentry when updating page in elastic
2022-03-28 10:30:31 -07:00
fd18affaec fix document_missing_exception being thrown to sentry when updating page in elastic 2022-03-28 21:16:33 +08:00
b6c04f3508 fix saving report item by linking to elastic page 2022-03-28 20:38:54 +08:00
1cdbfaca26 Merge pull request #294 from omnivore-app/feature/add-created-or-updated_at-to-newsletter_email
add created_at and updated_at to newsletter emails and sort by create…
2022-03-25 09:10:54 -07:00
5036924b21 Merge pull request #280 from omnivore-app/dependabot/npm_and_yarn/jsdom-19.0.0
Bump jsdom from 16.7.0 to 19.0.0
2022-03-25 09:08:55 -07:00
27735f7310 add page id in updated page json file in gcs (#319) 2022-03-25 13:50:43 +08:00
d595d4a8c4 add success and failure count in assertions (#318) 2022-03-25 12:16:43 +08:00
e7203bebb5 add a function_resolver to return originalArticleUrl from article.url (#317)
* add a function_resolver to return originalArticleUrl from article.url

* add a test
2022-03-25 11:25:30 +08:00
0264bc6733 Merge pull request #283 from omnivore-app/dependabot/npm_and_yarn/sentry/integrations-6.19.1
Bump @sentry/integrations from 5.30.0 to 6.19.1
2022-03-23 20:05:09 -07:00
3ec6f22199 Bump jsdom from 16.7.0 to 19.0.0
Bumps [jsdom](https://github.com/jsdom/jsdom) from 16.7.0 to 19.0.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/master/Changelog.md)
- [Commits](https://github.com/jsdom/jsdom/compare/16.7.0...19.0.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-23 18:51:08 +00:00
79a7eb11a3 fix data loss by selecting data from links instead of pages 2022-03-23 17:27:55 +08:00
8a485af20d validate publishedAt before inserting to elastic 2022-03-23 14:30:33 +08:00
e1752bb176 uncomment testing code 2022-03-23 12:37:33 +08:00
1b0399f286 add assertion of migrated data in python script 2022-03-23 12:09:25 +08:00
6597cc37b6 add created_at and updated_at to newsletter emails and sort by created_at desc 2022-03-22 18:43:47 +08:00
acec3b1acb Use lower priority queue for large imports
* add a count query in elastic
  * if priority is not set, check the rate limit of user's libary
    - if 5 articles added in the last minute: use low queue
    - default: use normal queue
2022-03-22 18:08:08 +08:00
7d19c933ac add elastic in README 2022-03-22 15:33:51 +08:00
1893e36375 Fix uploading endpoint returns 504 (#289)
* add timeout = 60s for uploading to private bucket

* add debug logs

* send response to client to close connection

* add tests for upload

* add PUBSUB_VERIFICATION_TOKEN in .env.test
2022-03-22 14:20:40 +08:00
3e9063c145 return empty content instead of null in search result (#291) 2022-03-22 13:34:44 +08:00
bee3b3c6fa fix article_id type mismatch (uuid => varchar) when saving highlights in postgres by setting article_id to be undefined (#285) 2022-03-22 11:48:01 +08:00
4ca7c89622 format date string as yyyy-mm-dd (#288) 2022-03-22 11:46:39 +08:00
c15efe6aad Fix sentry log size error (#287)
* reduce sentry log size error by partially logging page created in elastic

* remove some debugging logs

* remove more debugging logs
2022-03-22 11:45:58 +08:00
783118a2e4 Bump @sentry/integrations from 5.30.0 to 6.19.1
Bumps [@sentry/integrations](https://github.com/getsentry/sentry-javascript) from 5.30.0 to 6.19.1.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/5.30.0...6.19.1)

---
updated-dependencies:
- dependency-name: "@sentry/integrations"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-21 21:19:05 +00:00
f9bce1932c Print the logging level on startup 2022-03-21 09:31:05 -07:00
77050905a5 retry max three times if update labels is conflicted 2022-03-18 10:39:11 +08:00
9d8d2e40d7 reduce time to save reading progress in elastic update 2022-03-18 10:29:01 +08:00
ac567f7e71 retry max three times if update is conflicted 2022-03-18 10:17:20 +08:00
7910565dbf Merge pull request #251 from omnivore-app/fix/jsonld-encoding
Use html decoding when getting values from fetched oembed
2022-03-17 16:53:21 -07:00
563131ea23 Fix/newsletter label not add (#256)
* default sort by savedAt to avoid recent read page to appear on top of the list

* revert lint on generated code

* fix elastic update script syntax error
2022-03-17 13:22:38 +08:00
a2ce98229e default sort by savedAt to avoid recent read page to appear on top of… (#254)
* default sort by savedAt to avoid recent read page to appear on top of the list

* revert lint on generated code
2022-03-17 10:54:00 +08:00
af037a2837 make readingProgress required in the elastic page data (#253)
* make readingProgress required in the elastic page data

* delete readingProgress from function_resolvers because we have stored them in elastic
2022-03-17 10:06:21 +08:00
ff1200f3a1 Use html decoding when getting values from fetched oembed
If we fetch oembed data from an external source, instead of
handling it in readabilityjs we need to html decode it.
2022-03-16 15:29:42 -07:00
89f3719ba3 Fix linting 2022-03-16 14:23:39 -07:00
0314d8cf17 Add some extra logging 2022-03-16 13:53:55 -07:00
dc27c0c7a4 Return 0 if looking up page reading progress fails 2022-03-16 13:33:36 -07:00