Commit Graph

92 Commits

Author SHA1 Message Date
4e582fb55d Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
2025-01-27 13:33:16 +01:00
88a7e8d85b fix tests 2024-04-04 12:17:15 +08:00
03f89d890e fix tests 2024-02-08 10:42:37 +08:00
d357d2b0af fix tests 2024-01-24 13:41:49 +08:00
628cfc64d1 fix: replace \n with new line in redis cert 2024-01-22 11:09:15 +08:00
c297a8dd4e feat: group fetch content tasks by url 2024-01-18 18:48:46 +08:00
150a456c35 replace redis client library with ioredis 2024-01-16 15:42:50 +08:00
b65ada2a55 fix: too many open redis connections 2024-01-10 11:55:58 +08:00
34e6f1b00a fix redis command server incompatibility 2024-01-09 16:26:54 +08:00
26f91b3601 Merge pull request #3340 from omnivore-app/fix/test
fix rss handler tests
2024-01-09 12:36:19 +08:00
3cfee2915a fix rss handler tests 2024-01-09 12:23:39 +08:00
131261431d add comment 2024-01-09 12:09:42 +08:00
0969e81de3 fix: catch rss parsing failure in a nested try-catch block 2024-01-09 12:04:14 +08:00
a2ab8327ca fix: increment feed fetch failed count if rss feed is failed to fetch 2024-01-09 11:22:20 +08:00
44bed67e35 add comment 2024-01-05 15:41:01 +08:00
ccd4f2d3f0 catch redis client error and continue the process 2024-01-05 15:20:42 +08:00
b54b696fab skip parsing blocked feeds stored in redis 2024-01-04 23:06:37 +08:00
6d48816c22 fix tests 2024-01-04 22:42:31 +08:00
08c4624fce save the url in redis for 8 hours so rss-feeder will not try to re-save it 2024-01-04 18:03:34 +08:00
0f774b6ae4 skip recently saved rss item 2024-01-04 17:36:12 +08:00
816416934f Move the rss content block check to the rss handler 2023-12-29 12:42:43 +08:00
c9b93a1180 Dont try to fetch content for some feed URLs like arvix which blocks fetches 2023-12-29 12:26:43 +08:00
e0b8e4ae2b Merge pull request #3250 from omnivore-app/fix/duplicate-rss-item
skip fetching re-published items in the feed to deduplicate
2023-12-15 17:42:38 +08:00
1b9590f010 add test cases 2023-12-15 17:29:29 +08:00
991f984b6f skip fetching re-published items in the feed to deduplicate 2023-12-15 13:48:43 +08:00
a257147fc5 fetch following item content only when required 2023-12-14 17:25:55 +08:00
bd3945fc7e Merge pull request #3164 from omnivore-app/feature/fetch-content-for-following
feat: fetch content for rss feed items in following folder
2023-11-28 12:07:17 +08:00
3824da2ae5 Merge pull request #3166 from omnivore-app/feature/scan-feeds-api
add scan feeds api to scan feeds from opml or html
2023-11-28 11:27:36 +08:00
87bd9d5b60 Add group membership, more debug 2023-11-25 12:59:03 +08:00
af5040a518 saving posts from telegram channel as rss items 2023-11-24 17:23:35 +08:00
fd781644f1 feat: fetch content for rss feed items in following folder 2023-11-23 18:03:25 +08:00
4e60396d17 save thumbnail of the following item 2023-11-20 17:28:34 +08:00
148f329770 use service base url 2023-11-16 11:54:34 +08:00
5057448c4f update following task handler url 2023-11-15 15:58:17 +08:00
c59b4ca97e Merge pull request #3054 from omnivore-app/feature/social-feeds
feature: following
2023-11-15 11:19:14 +08:00
17fdbcca2c Some servers look for Accept: text/html for RSS 2023-11-10 21:01:57 +08:00
adf13131db add folder to library_item table 2023-11-10 17:34:46 +08:00
9dfc5f856d rename variables 2023-11-09 19:00:57 +08:00
f96ba17c92 fetch rss content if required 2023-11-09 16:09:23 +08:00
b7496db56c add following handler to save following item 2023-11-09 15:53:44 +08:00
a71181982f create a following task for each rss item fetched 2023-11-09 13:18:24 +08:00
a00ce8893e reduce feed parsing 2023-11-08 18:16:48 +08:00
9b0b378c39 check is_fetching_content and only fetch content if required for rss feed items 2023-11-08 18:16:48 +08:00
41805d13db Bump @sentry/serverless from 6.19.3 to 7.77.0
Bumps [@sentry/serverless](https://github.com/getsentry/sentry-javascript) from 6.19.3 to 7.77.0.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/6.19.3...7.77.0)

---
updated-dependencies:
- dependency-name: "@sentry/serverless"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-07 03:35:00 +00:00
d50726f3b4 fix: check lastBuildDate instead of pubDate in rss feed 2023-10-27 21:42:20 +08:00
b8eb376226 fix: wrong node version in docker-file in rss-handler 2023-10-27 16:01:27 +08:00
c4773dc904 Landing page improvements and various supporting improvements 2023-10-24 09:43:39 +01:00
3fa3526a16 fix wrong order of parameters in update scription api integration 2023-10-20 15:16:02 +08:00
25c0ee3b34 return 1 for updateFrequency if it is not a number 2023-10-19 21:08:54 +08:00
1880f2ace4 Update rss handler to fetch a group of rss feeds in one task 2023-10-19 17:27:55 +08:00