Benature/omnivore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)

Browse Source 
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
This commit is contained in:
Tom Rogers
2025-01-27 13:33:16 +01:00
committed by GitHub
parent 4cd5f2f02a
commit 4e582fb55d
339 changed files with 14859 additions and 11964 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
pkg/bull-queue-admin/Dockerfile
View File

@ -1,4 +1,4 @@
FROM node:18.16
FROM node:22.12
WORKDIR /app

4
pkg/discord/Dockerfile
View File

@ -1,4 +1,4 @@
FROM node:18.16 as builder
FROM node:22.12 as builder
WORKDIR /app
@ -17,7 +17,7 @@ COPY /packages/discord/tsconfig.json ./packages/discord/tsconfig.json
RUN yarn install --pure-lockfile
RUN yarn workspace @omnivore/discord build
FROM node:18.16 as runner
FROM node:22.12 as runner
WORKDIR /app

2
pkg/discord/package.json
View File

@ -14,7 +14,7 @@
"@google-cloud/pubsub": "^4.0.0",
"discord.js": "^14.14.1",
"dotenv": "^16.4.5",
"typescript": "^5.4.2",
"typescript": "^5.7.3",
"typescript-eslint": "^7.1.1"
},
"author": "",

2
pkg/extension/src/manifest.json
View File

@ -11,7 +11,7 @@
"url": "https://omnivore.app/"
},
"homepage_url": "https://omnivore.app/",
"content_security_policy": "default-src 'none'; child-src 'none'; manifest-src 'none'; media-src 'none'; object-src 'none'; worker-src 'none'; connect-src https://storage.googleapis.com/ process.env.OMNIVORE_GRAPHQL_URL blob:; frame-src 'none'; font-src 'none'; img-src data:; script-src 'self'; script-src-elem 'self' 'unsafe-inline'; script-src-attr 'unsafe-inline'; style-src 'self' 'unsafe-inline'; style-src-elem 'self' 'unsafe-inline'; style-src-attr 'none'; base-uri 'none'; form-action 'none'; block-all-mixed-content; upgrade-insecure-requests; report-uri https://api.jeurissen.co/reports/csp/webext/omnivore/",
"content_security_policy": "default-src 'none'; child-src 'none'; manifest-src 'none'; media-src 'none'; object-src 'none'; worker-src 'none'; connect-src *; frame-src 'none'; font-src 'none'; img-src data:; script-src 'self'; script-src-elem 'self' 'unsafe-inline'; script-src-attr 'unsafe-inline'; style-src 'self' 'unsafe-inline'; style-src-elem 'self' 'unsafe-inline'; style-src-attr 'none'; base-uri 'none'; form-action 'none'; block-all-mixed-content; upgrade-insecure-requests; report-uri https://api.jeurissen.co/reports/csp/webext/omnivore/",
"icons": {
"16": "/images/extension/icon-16.png",
"24": "/images/extension/icon-24.png",

7
pkg/extension/src/scripts/api.js
View File

@ -1,8 +1,9 @@
function gqlRequest(apiUrl, query) {
return getStorageItem('apiKey')
.then((apiKey) => {
return Promise.all([getStorageItem('apiKey'), getStorageItem('apiUrl')])
.then(([apiKey, savedUrl]) => {
const auth = apiKey ? { Authorization: apiKey } : {}
return fetch(apiUrl, {
const url = savedUrl ? `${savedUrl}/graphql` : apiUrl
return fetch(url, {
method: 'POST',
redirect: 'follow',
credentials: 'include',

43
pkg/extension/src/scripts/options.js
View File

@ -34,6 +34,40 @@ function clearAPIKey() {
})
}
function saveAPIUrl() {
var apiUrl = document.getElementById('api-url').value
if (!apiUrl) {
alert('No API URL specified.')
return
}
setStorage({
apiUrl: apiUrl,
}).then(() => {
alert('API URL saved!')
})
}
function loadAPIUrl() {
getStorageItem('apiUrl').then((apiUrl) => {
if (apiUrl) {
document.getElementById('api-url').value = apiUrl
} else {
alert('No API URL found in storage.')
}
})
}
function clearAPIUrl() {
document.getElementById('api-url').value = ''
setStorage({
apiUrl: undefined,
}).then(() => {
alert('API URL cleared!')
})
}
function autoDismissChanged(event) {
const value = document.getElementById('disable-auto-dismiss').checked
console.log(
@ -91,6 +125,15 @@ function handleConsent() {
document
.getElementById('clear-api-key-btn')
.addEventListener('click', clearAPIKey)
document
.getElementById('save-api-url-btn')
.addEventListener('click', saveAPIUrl)
document
.getElementById('load-api-url-btn')
.addEventListener('click', loadAPIUrl)
document
.getElementById('clear-api-url-btn')
.addEventListener('click', clearAPIUrl)
getStorageItem('disableAutoDismiss').then((value) => {
document.getElementById('disable-auto-dismiss').checked = value

18
pkg/extension/src/views/options.html
View File

@ -33,6 +33,24 @@
<p>&nbsp;</p>
<h1>API URL</h1>
<p>For users self hosting their own instance of Omnivore, you can set the API URL here.
Note this should be the URL of the graphql endpoint, not the webserver.
</p>
<p>
For example, if you are running the Omnivore API at <a href="">https://api-omnivore.mydomain.com</a>,
the API URL would be <a href="">https://api-omnivore.mydomain.com/api</a>.
</p>
<label for="api-url">API URL:</label>
<input type="text" id="api-url">
<br><br>
<button id="save-api-url-btn">Save API URL</button>
<button id="load-api-url-btn">Load API URL</button>
<button id="clear-api-url-btn">Clear API URL</button>
<p>&nbsp;</p>
<h1>Settings</h1>
<p>