Commit Graph

66 Commits

Author SHA1 Message Date
931c519d44 typeahead searchs by siteName 2022-07-13 10:37:06 +08:00
dc9523e522 add searchAsYouType in elastic 2022-07-12 09:36:00 -07:00
9feeb80cf3 Allow fetching original content
This is useful when debugging
2022-07-09 14:35:14 -07:00
41f43b54ab Publish delete event 2022-07-07 12:31:46 +08:00
efd47f3f83 add function to delete pages by param 2022-07-07 11:07:00 +08:00
c08afe6451 Update page updatedAt when updating labels 2022-06-28 12:03:37 +08:00
00f38d1134 Update page updatedAt when updating highlights 2022-06-28 11:55:26 +08:00
59efc5fea7 Add test 2022-06-28 10:14:15 +08:00
bf1faaeea4 Include highlights in search page result in elastic 2022-06-28 09:56:39 +08:00
e96d5552fa Fix search by is:read or is:unread getting empty result by using the correct property name 2022-06-23 12:35:48 +08:00
9a2f4b300a Fix a script error by deleting both labels in pages and highlights 2022-06-20 19:06:50 +08:00
f88083e535 Merge pull request #813 from omnivore-app/advanced-search
advanced search
2022-06-19 10:55:18 +08:00
26e7ab178e Fetch page savedAt, readingProgress from elastic 2022-06-17 17:08:26 +08:00
1be9c980e1 Add savedAt to searchItem and make readingProgress required in elastic 2022-06-17 16:03:49 +08:00
e3c30dd076 Allow matching title, description, content keyword in elastic 2022-06-16 22:15:05 +08:00
40bb49012a Allow match keyword 2022-06-16 22:15:01 +08:00
c5c8763767 Add readAt to searchItem 2022-06-15 09:32:53 +08:00
9baeb89742 Add more elastic debugging 2022-06-09 13:52:49 -07:00
520ce0077b Add refresh index method 2022-06-09 12:34:22 +08:00
4cf45af669 Add debug log for deleting label 2022-06-09 12:33:42 +08:00
309c581cf5 Merge pull request #729 from omnivore-app/labels-in-highlights
Support labels in highlights
2022-06-09 09:28:43 +08:00
2dcab66af2 Update labels for both pages and highlights 2022-06-08 17:39:38 +08:00
7c840f5b1e Merge deleteLabel in page and highlight 2022-06-08 17:25:33 +08:00
a981e13c86 Add test case for deleteLabel 2022-06-08 17:01:38 +08:00
22c604c15b Add deleteLabelForHighlights 2022-06-08 15:33:43 +08:00
bdda54d54a Add set Labels For Highlight data access method 2022-06-06 19:15:03 +08:00
577c6e84fb Add readAt to page type 2022-06-04 22:11:14 +08:00
04f7fdc31f Add labels in highlight type 2022-06-01 19:01:26 +08:00
195074fa49 Update search query 2022-06-01 13:49:51 +08:00
a04bfbb83e When searching for highlights we should include document title, description, content as secondary indexes 2022-05-29 22:43:21 +08:00
0ef9bb3944 Allow search by language in elastic 2022-05-26 10:55:20 +08:00
87b0f683e1 Add language term in search body 2022-05-26 10:55:04 +08:00
a702993485 Add search by language query 2022-05-26 10:52:18 +08:00
3e0e8b486f Add language in searchitem 2022-05-26 10:52:18 +08:00
be91562ca1 Add language in page 2022-05-26 10:52:18 +08:00
61065efd2b Add published sort param in elastic query 2022-05-24 18:24:30 +08:00
34349e925c Add unit test 2022-05-19 16:36:35 +08:00
b8225fc8c8 Fix labels not being updated in elastic 2022-05-19 12:08:50 +08:00
f51dbafdfc Stop throwing document not found exception to sentry 2022-05-13 11:40:08 +08:00
afe8b6e948 make savedAt a required field in page 2022-05-04 17:54:59 +08:00
9048ccbab3 Show failed pages in library 2022-05-04 12:46:57 +08:00
2b70d480d2 Remove article saving request (#493)
* Add state and taskName in elastic page mappings

* Add state and taskName in elastic page interface

* Create page with PROCESSING state before scrapping

* Update createArticleRequest API

* Fix tests

* Add default state for pages

* Update createArticle API

* Update save page

* Update save file

* Update saving item description

* Show unable to parse content for failed page

* Fix date parsing

* Search for not failed pages

* Fix tests

* Add test for saveUrl

* Update get article saving request api

* Update get article test

* Add test for articleSavingRequest API

* Add test for failure

* Return new page id if clientRequestId empty

* Update clientRequestId in savePage

* Update clientRequestId in saveFile

* Replace article with slug in articleSavingRequest

* Add slug in articleSavingRequest response

* Depreciate article

* Use slug in web

* Remove article and highlight fragments

* Query article.slug on Prod

* Show unable to parse description for failed page

* Fix a bug having duplicate pages when saving the same url multiple times

* Add state in response

* Rename variables in removeArticle API

* Rename state

* Add state in response in web

* Make state an enum

* Open temporary page by link id

* Use an empty reader view as the background for loading pages

* Progressively load the article page as content is loaded

* Add includePending flag in getArticles API

* Set includePending = true in web

* Add elastic update mappings in migration script

* Add elastic mappings in docker image

* Move index_settings.json to migrate package

* Remove elastic index creation in api

* Move elastic migrations to a separate directory

* Remove index_settings from api docker image

Co-authored-by: Jackson Harper <jacksonh@gmail.com>
2022-04-29 13:41:06 +08:00
71aa66ea9c Allow search for subscription:name (#461) 2022-04-22 08:47:41 +08:00
5c7f4a22bf Save subscriptions when saving newsletters (#447)
* parse unsub mailTo or httpUrl and send to pubsub

* save subscription from newsletter

* fix tests
2022-04-20 09:24:40 +08:00
d7a2659fc4 fix searching for multiple labels (#444) 2022-04-19 12:13:47 +08:00
c0966d908f upload highlight and label to gcs if successfully updated in elastic (#420) 2022-04-14 12:14:15 +08:00
0b0b521fab use generic method for uploading 2022-04-13 22:21:20 +08:00
a806f55233 upload highlight to gcs 2022-04-13 21:18:58 +08:00
ae0d1dd2ee Feature/search highlights backend (#395)
* add highlight mappings

* return highlight in resolvers

* temporarily skip highlight tests

* add test for getting highlights

* update merge highlight

* separate elastic methods

* roll back merge highlight test

* add highlight to elastic script

* update delete highlight in elastic

* migrate highlight data from postgres to elastic

* rescue not found exception when page is not found in the migration script

* exclude highlights in searching pages results

* search pages with highlights only with has:highlight query

* add search endpoint to search pages or highlights

* reduce code smell in search api

* fix rebase error

* fix tests

* add test for search highlight

* add test for new search endpoint

* add labels to search results

* update schema

* update search query

* fix update/share highlights

* fix rebase error

* fix tests

* add highlight model in elastic

* add savedAt and publishedAt date range in search query

* add sort by updated and recently read

* fix tests

* close db connection when tests are done

* test github action

* revert github action test

* fix rebase error

* add docker-compose for api-test

* remove unused env

* remove highlights with no page attached to

* allow get_articles resolver to search for query so we can merge it without web changes
2022-04-12 12:31:08 +08:00
6d405432af add site_name and site_icon to page model and return in resolver (#341)
* add site_name and site_icon to page model and return in resolver

* fix tests
2022-03-30 10:43:10 +08:00