Commit Graph

232 Commits

Author SHA1 Message Date
4e582fb55d Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
2025-01-27 13:33:16 +01:00
cbdd40fe74 Allow deleted articles to be read and restored from the article page 2024-08-30 13:01:12 +08:00
70d6e9dcae reduce logs 2024-08-29 18:18:17 +08:00
160d690589 fix: do not select content from database for update since api 2024-08-19 10:21:24 +08:00
19ed5501f6 fix: select readable_content from library_item table if content is requested by search api 2024-08-19 10:09:42 +08:00
626c456364 fix: select readable_content from library_item table if content is requested by update since api 2024-08-19 10:00:38 +08:00
26c7e95a00 update folder 2024-08-16 16:50:46 +08:00
3a07b21ec7 perf: stop serving content in search and update since api 2024-08-15 21:09:40 +08:00
cff44b83a8 fix: total count cache should be fixed for pagination 2024-08-14 16:10:41 +08:00
4bf3e4ba0f delete count cache after bulk action or empty trash 2024-08-14 12:59:52 +08:00
bcdff19a3d cache total count of library items in page info only 2024-08-14 09:40:59 +08:00
c2ec95845e request count of library items only when needed 2024-08-14 09:40:59 +08:00
4de6cfc1f8 fix: return markdown content for the old clients when highlighted markdown is requested 2024-08-07 09:34:35 +08:00
322f736fe0 stop storing original html in the database 2024-07-31 19:14:38 +08:00
465e5dc7d1 Add an index of columns user_id and created_at to the library_item table for counting query 2024-07-25 11:18:52 +08:00
6f12b5e406 reduce bulk action batch size to 50 2024-07-25 10:54:21 +08:00
822f249fb1 cont 2024-06-27 19:10:50 +08:00
c0947ddd48 fix batch delete query does not use index 2024-06-14 12:21:20 +08:00
aea1650626 do not show move/archived/deleted item in home feed 2024-06-14 12:15:17 +08:00
c620ef38a2 clean up helpers 2024-06-06 12:08:15 +08:00
91c2ae8d97 add refresh home api 2024-05-29 17:24:37 +08:00
a3fe526c2c remove debugging timers 2024-05-20 11:26:06 +08:00
eaf5632d0a profile query time 2024-05-19 10:17:50 +08:00
01ebcbb16b add bulk upload original content job 2024-05-10 14:37:05 +08:00
32be8126df reduce number of db calls for search library items 2024-04-30 12:48:19 +08:00
ef2ff36f0e fetch api key once 2024-04-28 21:51:30 +08:00
a78d438e21 fix: findOne with relations will execute query twice 2024-04-28 15:58:55 +08:00
7ef6973a47 load labels when getting item 2024-04-28 14:25:11 +08:00
fbe816d2d2 convert content if requested 2024-04-26 11:07:41 +08:00
2904c321e9 batching highlights 2024-04-25 23:03:08 +08:00
8501509564 include content in updateSince api response by default for offline use 2024-04-19 14:26:17 +08:00
09ce46e7ea include readable content in search api response by default 2024-04-19 11:30:09 +08:00
7567e3d0f6 * return only count if size is 0 because limit 0 is not allowed in typeorm 2024-02-28 16:57:15 +08:00
d26a2f403d replace segment with posthog 2024-02-26 16:47:04 +08:00
242e6e0fa3 delete all the labels and highlights attached to the item when item was deleted 2024-02-21 13:52:39 +08:00
d7a81483cf resolve conflicts 2024-02-06 17:02:00 +08:00
1b60694579 lint 2024-02-06 16:54:44 +08:00
7e022e3294 reduce find user query 2024-02-06 16:53:32 +08:00
a37bfab348 update create article api 2024-02-06 16:53:32 +08:00
4eba7df84c always upsert library items 2024-02-06 16:53:32 +08:00
d77fad804a fix tests 2024-02-05 18:29:58 +08:00
37c75865d1 do not update db immediately for force update of reading progress 2024-02-05 16:56:48 +08:00
54a4022409 reduce size 2024-02-02 15:45:11 +08:00
26b7103f0b improve batch update 2024-02-02 13:13:18 +08:00
2d1eebfad8 remove size 2024-02-02 12:21:08 +08:00
1bccf33320 add job_id and parent for tracking 2024-02-02 10:50:07 +08:00
2a0e6f8fa4 fix tests 2024-02-01 17:44:32 +08:00
702bd6c3c6 async bulk action on item count > 100 and create batch jobs 2024-02-01 17:44:30 +08:00
0e37bfa03d We need to update readAt because that isnt resolved from the cache 2024-02-01 09:59:42 +08:00
c47a720fe4 Do not cache on write, so the most updated item is always available on read
This is mostly an issue on tests where all the queries can be run
in a single process, but if we cache on write, it means the next
read will just have the cached value, instead of the calculated
maximum value.
2024-02-01 09:59:42 +08:00