4e582fb55d
* fix: Library Header layout shift * Bump Github Actions versions. * Self-Hosting Changes * Fix Minio Environment Variable * Just make pdfs successful, due to lack of PDFHandler * Fix issue where flag was set wrong * Added an NGINX Example file * Add some documentation for self-hosting via Docker Compose * Make some adjustments to Puppeteer due to failing sites. * adjust timings * Add start of Mail Service * Fix Docker Files * More email service stuff * Add Guide to use Zapier for Email-Importing. * Ensure that if no env is provided it uses the old email settings * Add some instructions for self-hosted email * Add SNS Endpoints for Mail Watcher * Add steps and functionality for using SES and SNS for email * Uncomment a few jobs. * Added option for Firefox for parser. Was having issues with Chromium on Docker. * Add missing space. Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com> * Fix some wording on the Guide * update browser extension to handle self-hosted instances * add slight documentation to options page * Fix MV * Do raw handlers for Medium * Fix images in Medium * Update self-hosting/GUIDE.md Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com> * Update Guide with other variables * Add The Verge to JS-less handlers * Update regex and image-proxy * Update self-hosting/nginx/nginx.conf Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com> * Update regex and image-proxy * Update self-hosting/docker-compose/docker-compose.yml Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com> * Fix Minio for Export * Merge to main * Update GUIDE with newer NGINX * Update nginx config to include api/save route * Enable Native PDF View for PDFS * Enable Native PDF View for PDFS * feat:lover packages test * feat:working build * feat:alpine build * docs:api dockerfile docs * Write a PDF.js wrapper to replace pspdfkit * Revert changes for replication, set settings to have default mode * build folder got removed due to gitignore on pdf * Add Box shadow to pdf pages * Add Toggle for Progress in PDFS, enabled native viewer toggle * Update node version to LTS * Update node version to LTS * Fix Linting issues * Fix Linting issues * Make env variable nullable * Add touchend listener for mobile * Make changes to PDF for mobile * fix(android): change serverUrl to selfhosted first * feat:2 stage alpine content fetch * feat:separated start script * fix:changed to node 22 * Add back youtube functionality and add guide * trigger build * Fix cache issue on YouTube * Allow empty AWS_S3_ENDPOINT * Allow empty AWS_S3_ENDPOINT * Add GCHR for all images * Add GCHR For self hosting. * Add GCHR For self hosting. * Test prebuilt. * Test prebuilt * Test prebuilt... * Fix web image * Remove Web Image (For now) * Move docker-compose to images * Move docker-compose files to correct locations * Remove the need for ARGS * Update packages, and Typescript versions * Fix * Fix issues with build on Web * Correct push * Fix Linting issues * Fix Trace import * Add missing types * Fix Tasks * Add information into guide about self-build * Fix issues with PDF Viewer --------- Co-authored-by: keumky2 <keumky2@woowahan.com> Co-authored-by: William Theaker <wtheaker@nvidia.com> Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com> Co-authored-by: David Adams <david@dadams2.com> Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com> Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com> Co-authored-by: Adil <mr.adil777@gmail.com>
Omnivore
Omnivore is a complete, open source read-it-later solution for people who like text.
We built Omnivore because we love reading and we want it to be more social. Join us!
- Highlighting, notes, search, and sharing
- Full keyboard navigation
- Automatically saves your place in long articles
- Add newsletter articles via email (with substack support!)
- PDF support
- Web app written in Node.js and TypeScript
- Native iOS app (source)
- Android app (source)
- Progressive web app for Android users
- Browser extensions for Chrome, Safari, Firefox, and Edge
- Labels (aka tagging)
- Offline support
- Text to speech (iOS only)
- Logseq support via our Logseq Plugin
- Obsidian support via our Obsidian Plugin
Every single part is fully open source! Fork it, extend it, or deploy it to your own server.
We also have a free hosted version of Omnivore at omnivore.app -- try it now!
Join us on Discord! 💬
We're building our community on Discord. Join us!
Read more about Omnivore on our blog. https://blog.omnivore.app/p/getting-started-with-omnivore
Shoutouts 🎉
Omnivore takes advantage of some great open source software:
- TypeScript - Most of our backend and frontend are written in TypeScript.
- Next.js - Our frontend is a Next.JS app and is hosted on Vercel.
- SWR - We do all our data fetching on the web using SWR.
- Stitches - We use Stitches on the frontend to style our components.
- Mozilla Readability - We use Mozilla's Readability library to make pages easier to read.
- Swift GraphQL - We generate our GraphQL queries on iOS using Swift GraphQL.
- Apollo GraphQL - We generate our GraphQL queries on Android using Apollo GraphQL.
- Radix - We use Radix UI's components on our frontend.
- And many more awesome libraries, just checkout our package files to see what we are using.
Importing Libraries
Check out our docs for information on importing your data from other apps.
How to setup local development 💻
The easiest way to get started with local development is to use docker compose up. This will start a postgres container, our web frontend, an API server, and our content fetching microservice.
Requirements for development
Omnivore is written in TypeScript and JavaScript.
- Node.js (v18.16) and Yarn -- Versions are managed by Volta.
- Chromium -- See below for installation info.
Running the web and API services
1. Start docker compose
git clone https://github.com/omnivore-app/omnivore
cd omnivore
docker compose up
This will start postgres, initialize the database, and start the web and api services.
2. Open the browser
Open http://localhost:3000 and confirm Omnivore is running
3. Login with the test account
During database setup docker compose creates an account demo@omnivore.app, password: demo_password.
Go to http://localhost:3000/ in your browser and choose Continue with Email to login.
Frontend Development
If you want to work on just the frontend of Omnivore you can run the backend services with docker compose and the frontend locally:
docker compose up api content-fetch
cd packages/web
cp .env.template .env.local
yarn dev
You will need to configure some values in the new .env.local file. These are
the values for running the web service directly on your host machine and
running api and content-fetch within docker:
NEXT_PUBLIC_BASE_URL=http://localhost:3000
NEXT_PUBLIC_HIGHLIGHTS_BASE_URL=http://localhost:3000
NEXT_PUBLIC_LOCAL_BASE_URL=http://localhost:3000
NEXT_PUBLIC_SERVER_BASE_URL=http://localhost:4000
NEXT_PUBLIC_LOCAL_SERVER_BASE_URL=http://localhost:4000
Running the puppeteer-parse service outside of Docker
To save pages you need to run the puppeteer-parse service.
1. Install and configure Chromium
brew install chromium --no-quarantine
export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
export CHROMIUM_PATH=`which chromium`
2. Navigate to the service directory, setup your env file, and install dependencies
cd packages/puppeteer-parse
cp .env.example .env
yarn
3. Start the service
yarn start
This will start the puppeteer-parse service on port 9090.
In your browser go to http://localhost:3000/home, click the Add Link button,
and enter a URL such as https://blog.omnivore.app/p/getting-started-with-omnivore.
You should see a Chromium window open and navigate to your link. When the service is done fetching your content you will see it in your library.
How to deploy to your own server
A guide for running a self hosted server can be found here
License
Omnivore and our extensions to Readability.js are under the AGPL-3.0 license.