Commit Graph

95 Commits

Author SHA1 Message Date
4e582fb55d Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
2025-01-27 13:33:16 +01:00
70d6e9dcae reduce logs 2024-08-29 18:18:17 +08:00
50ae288c0d use default remove-on-complete settings from the queue for saving newsletters tasks 2024-08-23 11:00:56 +08:00
4ebb555f55 keep up to 1000
completed newsletter tasks within a day
2024-08-22 21:01:19 +08:00
55b3e83ae6 add debugging logs for inbound-email-handler and keep email job data once completed or failed 2024-08-22 11:23:29 +08:00
b1ba2d3f26 fix Dockerfile 2024-07-02 15:25:51 +08:00
edefa988e7 fix tests 2024-07-02 15:00:51 +08:00
0d15833511 update inbound-email-handler to use @omnivore/utils to create redis connection 2024-07-02 14:59:42 +08:00
fd2f60ae3d fix dependecies 2024-04-11 11:48:50 +08:00
66c1f86ab5 fix dependecies 2024-04-11 11:28:40 +08:00
a062152d4a fix tests 2024-04-11 11:08:22 +08:00
07a14e0a7f fix tests 2024-04-11 11:08:22 +08:00
f66d3fa36c migrate inbound-email tasks from pubsub to bullmq 2024-04-11 11:08:22 +08:00
764b05fff4 save reply-to in recent emails 2024-04-11 11:08:22 +08:00
88a7e8d85b fix tests 2024-04-04 12:17:15 +08:00
809c572ca7 reduce the size of the log entry by truncating any string values to 500 characters 2024-01-29 14:32:45 +08:00
585957469c fix: init sentry sdk to disable tracing warning in cloud logs 2024-01-23 12:46:51 +08:00
41805d13db Bump @sentry/serverless from 6.19.3 to 7.77.0
Bumps [@sentry/serverless](https://github.com/getsentry/sentry-javascript) from 6.19.3 to 7.77.0.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/6.19.3...7.77.0)

---
updated-dependencies:
- dependency-name: "@sentry/serverless"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-07 03:35:00 +00:00
c4773dc904 Landing page improvements and various supporting improvements 2023-10-24 09:43:39 +01:00
9021854be7 convert plain text newsletter email content to html using markdown-to-html and save it if html is not available 2023-09-25 21:16:43 +08:00
792cf0b207 upgrade google cloud sdk 2023-08-11 13:55:29 +08:00
fa1ff9ba17 Upgrade node to 18.16 2023-07-24 15:55:11 +08:00
b891e547c9 Make content optional 2023-04-24 21:03:39 +08:00
187ad654e7 Add debug logs 2023-04-22 12:29:46 +08:00
50f55501f4 Fix list-unsubscribe header undefined 2023-04-21 15:42:58 +08:00
de0fb8542a Decode email metadata from rfc2047 encoded 2023-04-20 20:52:43 +08:00
8e3d2b0b91 Save epub email attachments 2023-04-19 11:36:28 +08:00
e48ba1c5b2 Add test case 2023-04-14 18:55:57 +08:00
182860e4d3 Update comment 2023-04-14 18:50:31 +08:00
0a2598e296 Parse the author name from email address 2023-04-14 18:48:24 +08:00
f7d46752e1 Forward subscription confirmation email 2023-04-14 18:40:38 +08:00
8c8538a49c Merge pull request #1936 from omnivore-app/dependabot/npm_and_yarn/google-cloud/pubsub-3.4.1
Bump @google-cloud/pubsub from 2.19.0 to 3.4.1
2023-04-13 10:51:50 +08:00
e620861eaf Invert newsletter detection logic 2023-04-06 18:07:10 +08:00
6ade830605 To in the envelope will always be a single-element array 2023-03-23 18:33:32 +08:00
ea3077ea2e Use receipient address in the envelope field 2023-03-23 18:18:33 +08:00
92c532a62a Bump @google-cloud/pubsub from 2.19.0 to 3.4.1
Bumps [@google-cloud/pubsub](https://github.com/googleapis/nodejs-pubsub) from 2.19.0 to 3.4.1.
- [Release notes](https://github.com/googleapis/nodejs-pubsub/releases)
- [Changelog](https://github.com/googleapis/nodejs-pubsub/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/nodejs-pubsub/compare/v2.19.0...v3.4.1)

---
updated-dependencies:
- dependency-name: "@google-cloud/pubsub"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 22:03:03 +00:00
017c8f5f8a Forward Gmail verification email to the user 2023-03-07 18:50:08 +08:00
88f6e20bfc Pass email headers to the content-handler 2023-01-30 18:52:38 +08:00
7dbf8662fc Fix test 2023-01-19 09:38:21 +08:00
8f9c317dd1 Save received email before publishing data 2023-01-18 22:35:07 +08:00
d60890d154 Add pubsub to save received email 2023-01-18 17:13:11 +08:00
3f373ab3b6 Save received email 2023-01-18 15:00:34 +08:00
66d3ffe7f3 Fix not correctly set confirmation code from Gmail in Spanish 2022-11-01 12:54:45 +08:00
d6e465d482 Add Dockerfile for pdfHandler 2022-10-04 15:28:12 +08:00
53d6afe25f Fix tests 2022-10-04 10:47:58 +08:00
9cae703666 Fix Dockerfile 2022-10-04 10:20:13 +08:00
78bce2d1e7 Add Dockerfile for cloud run 2022-10-04 09:51:42 +08:00
726df26c15 Temporarily copy content-handler to cloud functions root dir when doing cloud build 2022-10-03 15:39:05 +08:00
a9607adfd3 Import content-handler as local dependency 2022-10-03 11:11:24 +08:00
b00a516737 Add other newsletter handlers 2022-09-30 12:51:23 +08:00