Benature/omnivore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,486 Commits 1 Branch 0 Tags
0f1c2f0656bd54cb5b00a4b4adfa9934100b3788
Commit Graph

51 Commits

Author SHA1 Message Date
Satindar Dhillon
7d7b51500c migration to allow user to delete themseleves 2022-07-06 09:38:24 -07:00
Hongbo Wu
2ef5d101bb Create index if not exists in migration 2022-06-17 22:07:41 +08:00
Hongbo Wu
309c581cf5 Merge pull request #729 from omnivore-app/labels-in-highlights 
Support labels in highlights
 2022-06-09 09:28:43 +08:00
Hongbo Wu
5d08384db3 Add readAt field to page properties 2022-06-04 22:10:55 +08:00
Hongbo Wu
c59cf9cf65 Add labels in highlight properties 2022-06-01 19:01:14 +08:00
Hongbo Wu
74054cbdbd Merge pull request #716 from omnivore-app/api-token 
Api tokens
 2022-05-31 14:22:23 +08:00
Hongbo Wu
b77b378221 Grant permissions of api_key table to omnivore_user 2022-05-27 18:35:40 +08:00
Hongbo Wu
a09588d3a7 Update generateApiKey resolver 2022-05-27 18:33:49 +08:00
Hongbo Wu
849adf84d4 Add entity class 2022-05-27 18:33:49 +08:00
Hongbo Wu
5c1c9fb6de Grant permissions of webhooks table to omnivore_user 2022-05-27 17:32:18 +08:00
Hongbo Wu
f61f61312c Add webhook entity 2022-05-26 14:50:10 +08:00
Hongbo Wu
a655275754 Add unique constraint on user_id and event_types 2022-05-26 14:50:10 +08:00
Hongbo Wu
fe09487a6a Add webhooks model 2022-05-26 14:49:27 +08:00
Hongbo Wu
b0ed10760e Normalize language 2022-05-26 10:52:18 +08:00
Hongbo Wu
037520a1c3 Add language in mappings 2022-05-26 10:52:18 +08:00
Hongbo Wu
6c78d6188b Lowercase characters when doing text analysis 2022-05-25 09:59:43 +08:00
Hongbo Wu
34349e925c Add unit test 2022-05-19 16:36:35 +08:00
Hongbo Wu
b9c2c1186b Fix batch_size=500 to avoid exceeding max memory limit and requests_per_second=250 to throttle update requests 
Fix batch_size=500 to avoid exceeding max memory limit and requests_per_second=250 to throttle update requests
 2022-05-05 21:10:49 +08:00
Hongbo Wu
7183ca2e89 Add request timeout = 30m in elastic client 2022-05-05 13:28:01 +08:00
Hongbo Wu
911afd77e5 Reduce requests per second and scroll_size 2022-05-05 12:40:44 +08:00
Hongbo Wu
96fbd84bbb Extend timeout 2022-05-05 09:02:56 +08:00
Hongbo Wu
cd9c6a40d9 Tune the update speed 2022-05-04 22:44:28 +08:00
Hongbo Wu
8f60a9f905 Add default state to migration script 2022-05-04 21:54:28 +08:00
Hongbo Wu
46aac8ac0a Match the env var name (#512) 2022-04-29 14:10:56 +08:00
Hongbo Wu
2b70d480d2 Remove article saving request (#493) 
* Add state and taskName in elastic page mappings

* Add state and taskName in elastic page interface

* Create page with PROCESSING state before scrapping

* Update createArticleRequest API

* Fix tests

* Add default state for pages

* Update createArticle API

* Update save page

* Update save file

* Update saving item description

* Show unable to parse content for failed page

* Fix date parsing

* Search for not failed pages

* Fix tests

* Add test for saveUrl

* Update get article saving request api

* Update get article test

* Add test for articleSavingRequest API

* Add test for failure

* Return new page id if clientRequestId empty

* Update clientRequestId in savePage

* Update clientRequestId in saveFile

* Replace article with slug in articleSavingRequest

* Add slug in articleSavingRequest response

* Depreciate article

* Use slug in web

* Remove article and highlight fragments

* Query article.slug on Prod

* Show unable to parse description for failed page

* Fix a bug having duplicate pages when saving the same url multiple times

* Add state in response

* Rename variables in removeArticle API

* Rename state

* Add state in response in web

* Make state an enum

* Open temporary page by link id

* Use an empty reader view as the background for loading pages

* Progressively load the article page as content is loaded

* Add includePending flag in getArticles API

* Set includePending = true in web

* Add elastic update mappings in migration script

* Add elastic mappings in docker image

* Move index_settings.json to migrate package

* Remove elastic index creation in api

* Move elastic migrations to a separate directory

* Remove index_settings from api docker image

Co-authored-by: Jackson Harper <jacksonh@gmail.com>
 2022-04-29 13:41:06 +08:00
Hongbo Wu
bcbda50daa Add basic UI for subscriptions 2022-04-22 12:17:09 -07:00
Hongbo Wu
dca6461b62 Add unsubscribe resolver (#453) 
* add unsubscribe resolver

* add subscription to articles api response

* revert yarn.lock changes
 2022-04-21 13:18:15 +08:00
Hongbo Wu
1117a0c575 Feature/subscription list resolver (#432) 
* add subscriptions table

* add listSubscriptions schema

* add listSubscriptions resolver
 2022-04-19 11:08:43 +08:00
Hongbo Wu
250118de45 fix missing index_settings.json file in api dockerfile for creating elastic index (#363) 2022-04-01 13:07:44 +08:00
Jackson Harper
d260ad4def Remove the update permission on the link_labels table 
This was added mistakenly and isn't needed.
 2022-03-30 12:23:22 -07:00
Jackson Harper
857df99654 Add UPDATE permission to omnivore_user 2022-03-30 12:11:25 -07:00
Jackson Harper
bb91396595 Use user level security on labels update 2022-03-30 11:56:06 -07:00
Jackson Harper
57a45adb5d Add RLS permissions on the label table 2022-03-30 11:34:26 -07:00
Jackson Harper
15cf0d07e9 Allow UPDATE on the labels table 
labels are updated when users change their colour or description.
 2022-03-30 10:56:01 -07:00
Hongbo Wu
b6c04f3508 fix saving report item by linking to elastic page 2022-03-28 20:38:54 +08:00
Hongbo Wu
6597cc37b6 add created_at and updated_at to newsletter emails and sort by created_at desc 2022-03-22 18:43:47 +08:00
Jackson Harper
56c45fd1f6 Postgres migrations for the Elastic backend migration 
These are cherry picked from the elastic branch so we can run them
and migrate data before moving to elastic as the primary page
backend.
 2022-03-15 14:00:22 -07:00
Jackson Harper
78cb194912 Add package file back to migration image 
This is needed when running via docker-compose
 2022-03-03 11:05:42 -08:00
Jackson Harper
cf9e470f74 Create trigger with a new name 2022-02-24 22:38:34 -08:00
Jackson Harper
ae467d44f9 Syntax 2022-02-24 22:31:34 -08:00
Jackson Harper
e8ac5ed0c1 Remove the OR REPLACE when creating the article trigger 2022-02-24 22:28:00 -08:00
Jackson Harper
3926cefc1f Add comment to search index query 2022-02-24 22:07:26 -08:00
Jackson Harper
df3a0e1640 Add the link domain to the search index 
This adds the domain of the saved link to the search index in
two formats, the full domain like www.omnivore.app and just
the secondary domain like `omnivore`.

This also adds some ranking to search, making title, description,
and domain more highly ranked than the content but search results
wont change as we still order by saved_at.
 2022-02-24 22:07:26 -08:00
Hongbo Wu
5651c92efe ingore case when checking name uniqueness 2022-02-24 14:44:49 +08:00
Hongbo Wu
138c9682cd update tables 2022-02-24 14:44:48 +08:00
Hongbo Wu
e74f334db9 update entity 2022-02-24 14:44:48 +08:00
Hongbo Wu
966295385a add sql 2022-02-24 14:44:48 +08:00
Hongbo Wu
d53e94aeb2 add tests for create article 2022-02-16 16:14:14 +08:00
Hongbo Wu
77ba7531f3 add tests for upload 2022-02-16 15:26:11 +08:00
Jackson Harper
5f81cd4729 Remove root pkg dependencies from db docker image 
Because this is just used to execute the migrations we can
use the non-locked versions here and don't need to pull in all
of the root workspace packages.
 2022-02-13 22:12:19 -08:00