Files
omnivore/.github/workflows/run-tests.yaml
Tom Rogers 4e582fb55d Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
2025-01-27 13:33:16 +01:00

116 lines
3.8 KiB
YAML

name: Run tests
on:
push:
branches:
- main
paths-ignore:
- 'apple/**'
- 'android/**'
pull_request:
branches:
- main
paths-ignore:
- 'apple/**'
- 'android/**'
env:
NEXT_PUBLIC_APP_ENV: prod
NEXT_PUBLIC_BASE_URL: http://localhost:3000
NEXT_PUBLIC_SERVER_BASE_URL: http://localhost:4000
NEXT_PUBLIC_HIGHLIGHTS_BASE_URL: http://localhost:3000
jobs:
run-code-tests:
name: Run Codebase tests
runs-on: ${{ github.repository_owner == 'omnivore-app' && 'ubuntu-latest-m' || 'ubuntu-latest' }}
services:
postgres:
image: ankane/pgvector
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432
redis:
image: redis
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 10
ports:
- 6379
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Initialize the database
run: |
cp ./packages/api/.env.test ./packages/api/.env
psql --host localhost --port ${{ job.services.postgres.ports[5432] }} --user postgres --password -c "CREATE DATABASE omnivore_test;"
env:
PGPASSWORD: postgres
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: 22.12.0
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "::set-output name=dir::$(source ~/.nvm/nvm.sh && yarn cache dir)"
- name: Install dependencies
run: |
source ~/.nvm/nvm.sh
yarn install --frozen-lockfile
- name: Database Migration
run: |
psql -h localhost -p ${{ job.services.postgres.ports[5432] }} -U postgres -c "CREATE USER app_user WITH ENCRYPTED PASSWORD 'app_pass';"
yarn workspace @omnivore/db migrate
psql -h localhost -p ${{ job.services.postgres.ports[5432] }} -U postgres -c "GRANT omnivore_user to app_user;"
env:
PG_HOST: localhost
PG_PORT: ${{ job.services.postgres.ports[5432] }}
PG_USER: postgres
PG_PASSWORD: postgres
PG_DB: omnivore_test
PGPASSWORD: postgres # This is required for the psql command to work without a password prompt
PG_EXTRA_OPTIONS: '-c lock_timeout=2000'
- name: TypeScript Build and Lint
run: |
source ~/.nvm/nvm.sh
yarn build
yarn lint
- name: Tests
run: |
source ~/.nvm/nvm.sh
yarn test
timeout-minutes: 10
env:
PG_HOST: localhost
PG_PORT: ${{ job.services.postgres.ports[5432] }}
PG_USER: app_user
PG_PASSWORD: app_pass
PG_DB: omnivore_test
PG_LOGGER: debug
REDIS_URL: redis://localhost:${{ job.services.redis.ports[6379] }}
MQ_REDIS_URL: redis://localhost:${{ job.services.redis.ports[6379] }}
build-docker-images:
name: Build docker images
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build the API docker image
run: 'docker build --file packages/api/Dockerfile .'
- name: Build the content-fetch docker image
run: 'docker build --file packages/content-fetch/Dockerfile .'
- name: Build the inbound-email-handler docker image
run: 'docker build --file packages/inbound-email-handler/Dockerfile .'
- name: Build the tts docker image
run: 'docker build --file packages/text-to-speech/Dockerfile .'