An Org-mode query language, including search commands and saved views
This package provides a query language for Org files. It offers two syntax styles: Lisp-like sexps and search engine-like keywords.
It includes three libraries: The =org-ql= library is flexible and may be used as a backend for other tools. The libraries =org-ql-search= and =helm-org-ql= (a separate package) provide interactive search commands and saved views.
The package =org-ql= may be installed directly from [[https://melpa.org/#/org-ql][MELPA]] or with other tools like [[https://framagit.org/steckerhalter/quelpa][Quelpa]].
After installation, you can use the commands without additional configuration. To use the functions and macros in your own Elisp code, use libraries =org-ql= and =org-ql-view=.
Installing with [[https://framagit.org/steckerhalter/quelpa][Quelpa]] is easy:
(use-package org-ql :quelpa (org-ql :fetcher github :repo "alphapapa/org-ql" :files (:defaults (:exclude "helm-org-ql.el"))))
** Helm support
The command =helm-org-ql= is available in the package =helm-org-ql=. It may be installed from MELPA, or with Quelpa, like so:
(use-package helm-org-ql :quelpa (helm-org-ql :fetcher github :repo "alphapapa/org-ql" :files ("helm-org-ql.el")))
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
Lisp code examples are in [[examples.org]].
** Commands :PROPERTIES: :TOC: ignore-children :END:
/Note: This command supports both sexp queries and [[#non-sexp-query-syntax][non-sexp queries]]./
Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variables:
~BUFFERS-FILES~: ~A~ list of buffers and/or files to search. Interactively, may also be:
~GROUPS~: An ~org-super-agenda~ group set. See variable ~org-super-agenda-groups~.
~NARROW~: When non-nil, don't widen buffers before searching. Interactively, with prefix, leave narrowed.
~SORT~: One or a list of ~org-ql~ sorting functions, like ~date~ or ~priority~.
Bindings: Keys bound in results buffer. + =r=: Refresh results. With prefix, prompt to adjust search parameters. + =v=: Show =transient= view dispatcher (like Magit's popups). + =C-x C-s=: Save query to variable ~org-ql-views~ (accessible with command ~org-ql-view~).
Note: The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
/Note: This command uses [[#non-sexp-query-syntax][non-sexp queries]]. It is available separately in the package =helm-org-ql=./
This command displays matches with Helm.
Choose and display a view stored in ~org-ql-views~.
Bindings: Keys bound in view buffer. + =g=, =r=: Refresh results. With prefix, prompt to adjust search parameters. + =v=: Show =transient= view dispatcher (like Magit's popups). + =C-x C-s=: Save query to variable ~org-ql-views~ (accessible with command ~org-ql-view~).
Show a sidebar window listing views stored in =org-ql-views= for easy access. In the sidebar, press =RET= or =mouse-1= to show the view at point, and press =c= to customize the view at point.
Show items in ~FILES~ from last ~DAYS~ days with timestamps of ~TYPE~. ~TYPE~ may be ~ts~, ~ts-active~, ~ts-inactive~, ~clocked~, ~closed~, ~deadline~, ~planning~, or ~scheduled~. =FILES= defaults to those returned by the function =org-agenda-files=.
Arguments: ~(query &key keep-previous (buffer (current-buffer)))~
Show a sparse tree for ~QUERY~ in ~BUFFER~ and return number of results. The tree will show the lines where the query matches, and any other context defined in ~org-show-context-detail~, which see.
~QUERY~ is an ~org-ql~ query sexp (quoted, since this is a function). ~BUFFER~ defaults to the current buffer. When ~KEEP-PREVIOUS~ is non-nil (interactively, with prefix), the outline is not reset to the overview state before finding matches, which allows stacking calls to this command. Runs ~org-occur-hook~ after making the sparse tree.
** Queries :PROPERTIES: :TOC: :include descendants :depth 1 :END: :CONTENTS: - [[#non-sexp-query-syntax][Non-sexp query syntax]] - [[#general-predicates][General predicates]] - [[#ancestordescendant-predicates][Ancestor/descendant predicates]] - [[#datetime-predicates][Date/time predicates]] :END:
An =org-ql= query is a Lisp expression which may contain arbitrary expressions, as well as calling certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query. When possible, certain built-in predicates are optimized away to whole-buffer regular expression searches, which are much faster to search for than testing the predicate on each heading.
Notes: + Bare strings like ~"string"~ are automatically converted to ~(regexp "string")~ predicates. + Standard numeric comparator function symbols (~<~, ~<=~, ~>~, ~>=~, ~=~ ) need not be quoted when passed as an argument to predicates which accept them. The resemblance to infix notation is coincidental.
*** Non-sexp query syntax
The command =org-ql-search= also accepts, and the command =helm-org-ql= only accepts, an alternative, non-sexp query syntax. The syntax is simple, and a few examples of queries in both syntaxes should suffice. By default, when multiple predicates are used, they are combined with boolean =and=.
| Sexp syntax | Non-sexp syntax | |-------------------------------------------------+----------------------------------------------| | ~(todo)~ | ~todo:~ | | ~(todo "SOMEDAY")~ | ~todo:SOMEDAY~ | | ~(todo "SOMEDAY" "WAITING")~ | ~todo:SOMEDAY,WAITING~ | | ~(ts :on today)~ | ~ts:on=today~ | | ~(ts-active :from "2017-01-01" :to "2018-01-01")~ | ~ts-active:from=2017-01-01,to=2018-01-01~ | | ~(clocked :on -1)~ | ~clocked:on=-1~ | | ~(heading "quoted phrase" "word")~ | ~heading:"quoted phrase",word~ | | ~(and (tags "book" "books") (priority "A"))~ | ~tags:book,books priority:A~ | | ~(src :lang "elisp" :regexps ("defun"))~ | ~src:defun,lang=elisp~ or ~src:lang=elisp,defun~ | | ~(and (tags "space") (not (regexp "moon")))~ | ~tags:space !moon~ | | ~(priority >= B)~ | ~priority:A,B~ |
Note that the =priority= predicate does not support comparators in the non-sexp syntax, so multiple priorities should be passed instead, as seen in the last example.
*** General predicates
Arguments are listed next to predicate names, where applicable.
*** Ancestor/descendant predicates
*** Date/time predicates
All of these predicates take optional keyword arguments ~:from~, ~:to:~, and ~:on~:
Argument values should be either a number of days (positive to look forward, or negative to look backward), a ~ts~ struct, or a string parseable by ~parse-time-string~ (the string may omit the time value).
The following predicates, in addition to the keyword arguments, can also take a single argument, a number, which looks backward or forward a number of days. The number can be negative to invert the direction.
** Functions / Macros :PROPERTIES: :TOC: :include descendants :END: :CONTENTS: - [[#agenda-like-views][Agenda-like views]] - [[#function-org-ql-block][Function: org-ql-block]] - [[#listing--acting-on-results][Listing / acting-on results]] - [[#caching][Caching]] - [[#function-org-ql-select][Function: org-ql-select]] - [[#function-org-ql-query][Function: org-ql-query]] - [[#macro-org-ql-deprecated][Macro: org-ql (deprecated)]] - [[#custom-predicates][Custom predicates]] - [[#macro-org-ql-defpred][Macro: org-ql-defpred]] :END:
*** Agenda-like views
**** Function: ~org-ql-block~
For use as a custom agenda block type in ~org-agenda-custom-commands~. For example, you could define a custom series command like this, which would list all priority A items tagged =Emacs= with to-do keyword =SOMEDAY=, followed by the standard agenda view, in a single buffer:
(setq org-agenda-custom-commands '(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items" ((org-ql-block '(and (todo "SOMEDAY") (tags "Emacs") (priority "A")) ((org-ql-block-header "SOMEDAY :Emacs: High-priority"))) (agenda)))))
Which would be equivalent to a ~tags-todo~ search like this:
(setq org-agenda-custom-commands '(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items" ((tags-todo "PRIORITY=\"A\"+Emacs/!SOMEDAY") (agenda)))))
However, the ~org-ql-block~ version runs in about 1/5th the time.
The variable =org-ql-block-header= may be bound to a string to use as the block header, otherwise the header is formed automatically.
*** Listing / acting-on results
Org QL uses a per-buffer cache to speed up subsequent searches. It's keyed on query expressions and match actions, which means that, for the same query and same match action in the same buffer, if the buffer has not been modified since the last time the query was run, the cached match-action result will be returned, and the query will not be evaluated in that buffer again.
Therefore, since neither query expressions nor match actions are guaranteed to be evaluated when the following functions are called, they should be free of side effects. Or, if a side effect is required, the cache should be invalidated (e.g. by incrementing the buffer's modified tick, or by using a query expression or match action that has yet to be cached). /Note: Future improvements will allow the cache to be more easily disabled or cleared./
**** Function: ~org-ql-select~
/Arguments:/ ~(buffers-or-files query &key action narrow sort)~
Return items matching ~QUERY~ in ~BUFFERS-OR-FILES~.
~BUFFERS-OR-FILES~ is a one or a list of files and/or buffers.
~QUERY~ is an ~org-ql~ query sexp (quoted, since this is a function).
~ACTION~ is a function which is called on each matching entry with point at the beginning of its heading. It may be:
~element~ or nil: Equivalent to ~org-element-headline-parser~.
~element-with-markers~: Equivalent to calling ~org-element-headline-parser~, with markers added using ~org-ql--add-markers~. Suitable for formatting with ~org-ql-agenda--format-element~, allowing insertion into an Org Agenda-like buffer.
A sexp, which will be byte-compiled into a lambda function.
A function symbol.
If ~NARROW~ is non-nil, buffers are not widened (the default is to widen and search the entire buffer).
~SORT~ is either nil, in which case items are not sorted; or one or a list of defined ~org-ql~ sorting methods (~date~, ~deadline~, ~scheduled~, ~todo~, ~priority~, or ~random~); or a user-defined comparator function that accepts two items as arguments and returns nil or non-nil.
;; Return list of to-do headings in inbox file with tags and to-do keywords: (org-ql-select "~/org/inbox.org" '(todo) :action #'org-get-heading) ;; => ("TODO Practice leaping tall buildings in a single bound :personal:" ...)
;; Without tags and to-do keywords: (org-ql-select "~/org/inbox.org" '(todo) :action '(org-get-heading t t)) ;; => ("Practice leaping tall buildings in a single bound" ...)
;; Return WAITING heading elements in agenda files: (org-ql-select (org-agenda-files) '(todo "WAITING") :action 'element) ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)
;; Since `element' is the default for ACTION, it may be omitted: (org-ql-select (org-agenda-files) '(todo "WAITING")) ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)
**** Function: ~org-ql-query~
/Arguments:/ ~(&key (select 'element-with-markers) from where order-by narrow)~
Like ~org-ql-select~, but arguments are named more like a ~SQL~ query.
;; Return list of to-do headings in inbox file with tags and to-do keywords: (org-ql-query :select #'org-get-heading :from "~/org/inbox.org" :where '(todo)) ;; => ("TODO Practice leaping tall buildings in a single bound :personal:" ...)
;; Without tags and to-do keywords: (org-ql-query :select '(org-get-heading t t) :from "~/org/inbox.org" :where '(todo)) ;; => ("Practice leaping tall buildings in a single bound" ...)
;; Return WAITING heading elements in agenda files: (org-ql-query :select 'element :from (org-agenda-files) :where '(todo "WAITING")) ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)
;; Since `element' is the default for SELECT, it may be omitted: (org-ql-query :from (org-agenda-files) :where '(todo "WAITING")) ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)
**** Macro: ~org-ql~ (deprecated)
/Arguments:/ ~(buffers-or-files query &key sort narrow markers action)~
Expands into a call to ~org-ql-select~ with the same arguments. For convenience, arguments should be unquoted.
/Note: This macro is deprecated and will be removed in v0.7./
*** Custom predicates
**** Macro: =org-ql-defpred=
/Arguments:/ ~(name args docstring &key body preambles normalizers)~
Define an ~org-ql~ selector predicate named ~org-ql--predicate-NAME~. ~NAME~ may be a symbol or a list of symbols: if a list, the first is used as ~NAME~ and the rest are aliases. ~A~ function is only created for ~NAME~, not for aliases, so a normalizer should be used to replace aliases with ~NAME~ in queries (keep reading).
~ARGS~ is a ~cl-defun~-style argument list. ~DOCSTRING~ is the function's docstring.
~BODY~ is the body of the predicate. It will be evaluated with point on the beginning of an Org heading and should return non-nil if the heading's entry is a match.
~PREAMBLES~ and ~NORMALIZERS~ are lists of ~pcase~ forms matched against Org ~QL~ query sexps. They are spliced into ~pcase~ forms in the definitions of the functions ~org-ql--query-preamble~ and ~org-ql--normalize-query~, which see. Those functions are redefined when this macro is expanded, unless variable ~org-ql-defpred-defer~ is non-nil, in which case those functions should be redefined manually after defining predicates by calling ~org-ql--define-query-preamble-fn~ and ~org-ql--define-normalize-query-fn~.
~NORMALIZERS~ are used to normalize query expressions to standard forms. For example, when the predicate has aliases, the aliases should be replaced with predicate names using a normalizer. Also, predicate arguments may be put into a more optimal form so that the predicate has less work to do at query time.
~PREAMBLES~ refer to regular expressions which may be used to search through a buffer directly to a potential match rather than testing the predicate body on each heading. (Naming things is hard.) In each ~pcase~ form in ~PREAMBLES~, the ~pcase~ expression (not the pattern) should be a plist with the following keys, each value of which should be an expression which may refer to variables bound in the pattern:
~:regexp~ Regular expression which searches directly to a potential match.
~:case-fold~ Bound to ~case-fold-search~ around the regexp search.
~:query~ Expression which should replace the query expression, or ~query~ if it should not be changed (e.g. if the regexp is insufficient to determine whether a heading matches, in which case the predicate's body needs to be tested on the heading). If the regexp guarantees a match, this may be simply ~t~, leaving the query expression with no work to do, which improves performance.
For convenience, within the ~pcase~ patterns, the symbol ~predicate-names~ is a special form which is replaced with a pattern matching any of the predicate's name and aliases. For example, if ~NAME~ were:
Then if ~NORMALIZERS~ were:
(,predicate-names . ,args)(heading ,@args)))~
It would be expanded to:
(,(or 'heading 'h) . ,args)(heading ,@args)))~
** Dynamic block
Org QL provides a dynamic block that lists entries in the current document matching a query. In the header, these parameters are supported:
The heading column is formatted as a link to the heading (not shown in the following example).
For example, this dynamic block shows the first seven headings that are to-do items with priority A or B, sorted by deadline then priority, with certain columns (including the value of the =agenda-group= property with a custom header) and timestamp format:
,#+BEGIN: org-ql :query "todo: priority:A,B" :columns (todo (priority "P") ((property "agenda-group") "Group") deadline heading) :sort (deadline priority) :take 7 :ts-format "%Y-%m-%d %H:%M" | Todo | P | Group | Deadline | Heading | |------+---+-------+------------------+---------------------------------------| | TODO | A | | 2017-07-07 00:00 | Take over the world | | TODO | B | | 2017-07-10 00:00 | Renew membership in supervillain club | | TODO | A | plans | 2017-07-15 00:00 | Take over the universe | | TODO | B | | 2017-07-21 00:00 | Internet | | TODO | A | bills | 2017-08-01 00:00 | Spaceship lease | | TODO | A | | | Skype with president of Antarctica | | TODO | B | | | Take over Mars | ,#+END:
Org QL View searches may be accessed by opening ~org-ql-search:~ links in an Org file.
In an Org QL View buffer, the command ~org-store-link~ (i.e. ~C-c l~) stores a link to the current search, and it may be inserted into an Org buffer with the command ~org-insert-link~ (~C-c C-l~). The stored link records all of the view settings, like title, sorting, and grouping.
Simple links may also be written manually in either sexp or non-sexp form, like:
[[org-ql-search:todo:NEXT priority:A]] [[org-ql-search:(and (todo "NEXT") (priority "A"))]]
Org QL View buffers can be bookmarked with Emacs bookmark commands, e.g. =C-x r m=. This also integrates with [[https://github.com/alphapapa/org-sidebar][org-sidebar]] and [[https://github.com/alphapapa/burly.el][Burly]].
Changelog :PROPERTIES: :TOC: ignore-children :END:
/Note:/ Breaking changes may be made before version 1.0, but in the event of major changes, attempts at backward compatibility will be made with obsolescence declarations, translation of arguments, etc. Users who need stability guarantees before 1.0 may choose to use tagged stable releases.
Added + Macro =org-ql-defpred=, used to define search predicates. (See [[file:examples/defpred.org][tutorial]].) + Predicate ~heading-regexp~, which matches regular expressions against heading text (alias: ~h*~).
Changed + Helm support (including the command =helm-org-ql=) has been moved to a separate package, =helm-org-ql=. + Predicate ~heading~ now matches plain strings instead of regular expressions.
Internal + Predicates are now defined more cleanly with a macro (=org-ql-defpred=) that consolidates functionality related to each predicate. This will also allow users to more easily define custom predicates. + Version 1.0 of library ~peg~ is now required.
Added + View dispatcher using =transient.el= (like Magit), bound to =v= in search/view buffers. + Predicate =link=, which matches descriptions and targets in Org links. + Predicate ~tags-regexp~ (alias: ~tags*~), which matches regexps against entry tags (e.g, helpful when a tag might end in "s"). + Emacs bookmark support: Org QL View buffers can be bookmarked with, e.g. =C-x r m= and shown with, e.g. =C-x r b=. (This also enables view restoration with [[https://github.com/alphapapa/burly.el][Burly]].) + Dynamic block support. + Org link support (storing and opening links to Org QL View searches). + Mascot.
Changed + Binding to refresh search/view buffers changed to =r=.
Internal + When formatting entries for Org QL View buffers, use internal function for retrieving heading tags. This improves speed by using our cache, and it removes the need for a compatibility alias for Org versions before 9.3.
Deprecated + Macro =org-ql= is marked obsolete. It will be removed in v0.7. Functions =org-ql-select= and =org-ql-query= should be used instead. (The macro serves only to confuse with regard to quoting arguments.)
Acknowledgments + [[https://github.com/tpeacock19][tpeacock19]] for extensive help testing new features in this version.
Fixed + Agenda restriction in =org-ql-block=. (Fixes [[https://github.com/alphapapa/org-ql/issues/84][#84]]. Thanks to [[https://github.com/yantar92][Ihor Radchenko]].)
Fixed + Multiple sorters not preserved when refreshing views. (Fixes [[https://github.com/alphapapa/org-ql/issues/136][#136]], [[https://github.com/alphapapa/org-ql/pull/137][#137]]. Thanks to [[https://github.com/natrys][Imran Khan]].)
Fixed + Give a useful error if =org-ql-search-directories-files= is called without a directories argument and =org-directory= doesn't exist. (Fixes [[https://github.com/alphapapa/org-ql/issues/139][#139]]. Thanks to [[https://github.com/matthuszagh][Matt Huszagh]] for reporting.)
Fixed + Compatibility with newer versions of the =peg= library, which removed a macro used by this package. (Fixes [[https://github.com/alphapapa/org-ql/issues/75][#75]]. Thanks to [[https://github.com/novoid][Karl Voit]] and [[https://github.com/karlicoss][@karlicoss]] for reporting.)
Fixed + Non-case-folding predicates like ~(todo)~ unnecessarily disabled case-folding for other predicates. ([[https://github.com/alphapapa/org-ql/issues/114][Issue #114]]. Thanks to [[https://github.com/bitclick][@bitclick]] for reporting.)
Fixed + Compatibility with Org Agenda remote editing commands (some of which were broken by 0.4.3). (Fixes [[https://github.com/alphapapa/org-ql/issues/102][#102]]. Thanks to [[https://github.com/AloisJanicek][Alois Janíček]] for reporting.)
Fixed + When =org-ql-view-refresh= is called, ensure the buffer is an Org QL View buffer.
Fixed + Items' to-do keywords were not shown in views.
Fixed + =level= predicate used with arguments in plain queries. (Thanks to [[https://github.com/akirak][Akira Komamura]] for reporting.)
/Note:/ The next release, 0.5, may include changes which will require minor updates to written queries (e.g. a few predicates may be renamed). Users who wish to avoid those changes happening unexpectedly in their configs should avoid upgrading =org-ql= beyond 0.4 automatically, as they will be pushed to the =master= branch when ready.
Added + Commands - ~helm-org-ql-views~, which shows one of ~org-ql-views~ selected with Helm. - ~org-ql-search~ can search files in ~org-directory~; customization options are available in the ~org-ql-search~ group. - ~org-ql-view-refresh~ can be called with a prefix argument to adjust search parameters. + Queries - Negation of terms in plain queries using ~!~. For example, ~tags:space !moon~ to exclude entries which contain ~moon~. - Predicates =outline-path= (alias =olp=) and =outline-path-segment= (alias =olps=). - Predicate ~src~, which matches Org Babel source blocks. - Predicates =parent= and =ancestors=. (Thanks to [[https://github.com/mm--][Josh Moller-Mara]].) - Alias =h= for =heading= predicate. - Alias =r= for =regexp= predicate. (Thanks to [[https://github.com/tumashu][Feng Shu]].) + Info manual. + Function ~helm-org-ql-source~, which returns a Helm source that searches given buffers/files with ~helm-org-ql~. It can be used for custom Helm commands that search certain files. + Display a message when views are refreshed. (Thanks to [[https://github.com/xeijin][xeijin]].) + Respect Org Agenda restriction in =org-ql-block=. (Thanks to [[https://github.com/yantar92][Ihor Radchenko]] for reporting.) + Option =org-ql-view-sidebar-sort-views=. + Mouseover =help-echo= text for =org-ql-views= default view names. + "Dangling tasks" default view in =org-ql-views=. (Users who have modified =org-ql-views= from the default will not see the new view unless they copy it into their config.)
Changed + Some default =org-ql-view= views (users who have modified =org-ql-views= from the default will not see the new views unless they copy them into their config): - Rename some views. - "Stuck projects" view (now uses =descendants= instead of =children=, which is more useful.
Fixed + Inherit file tags when =org-tag-inheritance= is enabled. (Fixes [[https://github.com/alphapapa/org-ql/issues/55][#55]]. Thanks to [[https://github.com/mskorzhinskiy][Mikhail Skorzhinskiy]].) + Call =helm-make-source= directly instead of using =helm-build-sync-source= macro. (Fixes [[https://github.com/alphapapa/org-ql/issues/60][#60]]. Thanks to [[https://github.com/matthuszagh][Matt Huszagh]] for reporting.) + Search/view buffers now always end with a newline, which prevents side-scrolling of the window when calling =end-of-buffer=. + Face for done to-do keywords in =org-ql-view= buffers. (Thanks to [[https://github.com/dsdshcym][Yiming Chen]].) + Make view buffers read-only. (Fixes [[https://github.com/alphapapa/org-ql/issues/72][#72]]. Thanks to [[https://github.com/xeijin][xeijin]].) + Sorting with single sorter specified as an atom. (Thanks to [[https://github.com/legalnonsense][Jeff Filipovits]].) + Autoload for =org-ql-block= agenda block. (Fixes [[https://github.com/alphapapa/org-ql/issues/53][#53]]. Thanks to reports from [[https://github.com/gcantieni][Gus Cantieni]], [[https://github.com/novoid][Karl Voit]], [[https://github.com/rieje][rieje]], and [[https://github.com/jakejx][Jake | Junxuan]].)
Internal + Added generic node data cache to speed up recursive, tree-based queries.
Fixed + In =org-ql-search=, accept symbol as ~:super-groups~ argument. + In the =This week= and =Next week= default =org-ql-views= views, set timestamps for beginning-of-week to 00:00:00 and end-of-week to 23:59:59. + Plain quoted-phrases in non-sexp queries.
Fixed + Compatibility with Org 9.2. Thanks to [[https://github.com/leungbk][Brian Leung]].
Added + Alternative, non-sexp query syntax for commands =org-ql-search= and =helm-org-ql=. See [[#non-sexp-query-syntax][documentation]]. + Command =helm-org-ql=. + Command =org-ql-sparse-tree=, like =org-sparse-tree= for =org-ql= queries. (Thanks to [[https://github.com/akirak][Akira Komamura]].) + Command =org-ql-view-sidebar=. + Per-buffer, per-heading tag caching, which increases the speed of tags-related queries by 6-7x. + More tags-related predicates and aliases: - For inherited tags: =tags-inherited=, =inherited-tags=, =tags-i=, =itags=. - For heading-local tags: =tags-local=, =local-tags=, =tags-l=, =ltags=. - =tags-all=, =tags&=: Matches all given tags using boolean =AND= (rather than boolean =OR=, which the =tags= predicate uses). + Variable =org-ql-block-header=, which overrides the default header in =org-ql-block= agenda blocks. + Predicate =(path)=. + Option =org-ql-views= may now be customized in a guided, structured way with the customization UI (e.g. =M-x customize-option RET org-ql-views RET=, or press =c= in the =org-ql-view-sidebar= buffer). + Enable more Org Agenda commands in =org-ql-view= buffers (e.g. setting deadlines and scheduling). (Fixes [[https://github.com/alphapapa/org-ql/issues/35][#35]]. Thanks to [[https://github.com/mz-pdm][Milan Zamazal]] and [[https://github.com/mskorzhinskiy][Mikhail Skorzhinskii]].) + Function =org-ql-select='s =buffers-files= argument can be a function which returns a list of buffers and/or files.
Changed + Predicate =heading= now accepts multiple regexps, which are matched with boolean =AND=. + Predicate =regexp= now matches its regexp arguments with boolean =AND=. + Package =org-super-agenda= is now a dependency. This removes the need for awkward code to handle the case where it's not installed, and makes grouping features always available. Of course, the global minor mode =org-super-agenda-mode= is not activated by =org-ql=, so no behavior is changed in Org Agenda or =org-ql=; it only means that commands like =org-ql-search= will always provide grouping when called with the appropriate arguments.
Removed + Macro =org-ql-agenda=. Instead, use function =org-ql-search=. See also command =org-ql-view=, etc.
Fixed + Predicate =heading= now matches only against heading text, i.e. not including tags at the end of the line, to-do keyword, etc. + Predicate =todo= now matches case-sensitively, avoiding non-todo-keyword matches (e.g. a heading which begins =Waiting on= will no longer match for a todo keyword =WAITING=). + Interactive completion in =org-ql-search=.
Internal + Refactored code from file =org-ql-agenda.el= into files =org-ql-search.el= and =org-ql-view.el=. Function and variable names have been changed accordingly.
Fixed + Priority queries could fail to match headings whose to-do keywords had non-alphabetic characters, like =TO-READ=.
Fixed + =(deadline auto)= selector matched entries whose deadlines had a warning period that had not yet been entered (=org-deadline-warning-days= too soon).
Fixed + =(descendants)= selector matched against parent heading instead of only descendants.
** 0.2 :PROPERTIES: :ID: 67be09f9-e959-4333-9be2-93ad8f458fbe :END:
Added + Function ~org-ql-query~, like ~org-ql-select~ but with arguments named more like a SQL query. + Bare strings like ~"string"~ can be used in queries, which are converted to ~(regexp "string")~ automatically. + Selector ~(regexp)~ accepts multiple regexps to test. + Macro ~org-ql~ and functions ~org-ql-query~ and ~org-ql-select~ now also accept a comparator function in their ~:sort~ argument. + Function ~org-ql-block~, which works as an Org Agenda series/composite/block command, usable in custom agenda commands defined in variable ~org-agenda-custom-commands~. (Inspired by [[https://github.com/pestctrl/emacs-config/blob/84c557982a860e86d6f67976a82ea776a7bd2c7a/config-org-new.org#my-own-agenda-renderer][Benson Chu's config]].) + Function ~org-ql-agenda--agenda~ optionally takes a list of entries as an argument. + Selectors ~ts-a~ and ~ts-i~, aliases for ~ts-active~ and ~ts-inactive~. + Selector ~ts~ now accepts a ~:type~ argument. + Face =org-ql-agenda-due-date=. + Selectors ~(children)~ and ~(descendants)~. + Function ~org-ql-search~ and macro ~org-ql-agenda~ accept a ~:title~ argument, which is displayed in the header. + Command ~org-ql-search~ offers global ~org-super-agenda-groups~ in completion. + Customization group ~org-ql~. + Command ~org-ql-view~, which displays views saved to variable ~org-ql-views~, which can be saved from ~org-ql-search~ buffers with command ~org-ql-search-save~, which is bound to =C-x C-s= in view buffers. + Variable ~org-ql-view-map~, active in view buffers displayed by ~org-ql-search~, ~org-ql-agenda~, and ~org-ql-view~. + =random= sort method. + Save position when refreshing search buffers.
Changed + Function ~org-ql-query~ renamed to ~org-ql-select~. ~org-ql-query~ now refers to a new function. + Macro ~org-ql~ no longer accepts a ~:markers~ argument. Instead, use argument ~:action element-with-markers~. See function ~org-ql-select~, which ~org-ql~ calls. + Selector ~(todo)~ no longer matches "done" keywords when used without arguments (i.e. the ones in variable ~org-done-keywords~). + Overhauled date/time-based predicates. See documentation for new argument signatures.
Removed + Selector ~(date)~, replaced by ~(ts)~.
Fixed + Handle date ranges in date-based selectors. (Thanks to [[https://github.com/codygman][Cody Goodman]], [[https://github.com/swflint][Samuel W. Flint]], and [[https://github.com/vikasrawal][Vikas Rawal]].) + Don't overwrite bindings in =org-agenda-mode-map=. + Don't search buffers without headings, and show a message if the user attempts it. + Don't search hidden/special buffers. + Properly accept arbitrary sort functions in =org-ql-select=, etc. (Fixes [[https://github.com/alphapapa/org-ql/issues/37][#37]]. Thanks to [[https://github.com/mz-pdm][Milan Zamazal]].) + Planning-line-related predicates searched too far into entries. + Add autoloads. (Fixes [[https://github.com/alphapapa/org-ql/pull/36/files#][#36]]. Thanks to [[https://github.com/akirak][Akira Komamura]].)
Compatibility + Fixes for compatibility with Org 9.2. (Thanks to [[https://github.com/ataias][Ataias Pereira Reis]] and [[https://github.com/dakra][Daniel Kraus]].)
Internal + Optimizations for some query selectors, e.g. =regexp= and =todo=. These can provide a significant improvement for some queries. See benchmarks in [[file:notes.org][notes.org]]. + Library [[https://github.com/alphapapa/ts.el][ts]] is now used for parsing and comparing timestamps.
First tagged release.
** Comparison with Org Agenda searches
Of course, queries like these can already be written with Org Agenda searches, but the syntax can be complex. For example, this query would be difficult to write in a standard Org Agenda search, because it matches against a to-do keyword /and/ a plain-text search. As described in the [[https://orgmode.org/worg/org-tutorials/advanced-searching.html#combining-metadata-and-full-text-queries][advanced searching tutorial]], it would require using ~org-search-view~ with a query with specific regular expression syntax, like this:
But with =org-ql-search=, you would write a query like =lisp todo:TO-READ=, or in Lisp syntax, ~(and "lisp" (todo "TO-READ"))~.
This package is used by [[https://github.com/alphapapa/org-sidebar][org-sidebar]], which presents a customizable agenda-like view in a sidebar window.
Code used to update this document.
Generates the predicate subtree.
(defun org-ql--readme-update-predicates () "Update predicate subtree in current document." (interactive) (org-ql--readme-replace-node '("Usage" "Queries" "Predicates") (org-ql--readme-predicate-list)))
(defun org-ql--readme-replace-node (outline-path string) "Replace contents of node at OUTLINE-PATH with STRING." (org-with-wide-buffer (-let* ((subtree-marker (org-find-olp outline-path t)) ((_headline element) (progn (goto-char subtree-marker) (org-element-headline-parser (point-max)))) ((&plist :contents-begin beg :contents-end end) element)) (goto-char beg) (delete-region (point) (1- end)) (insert string "\n"))))
(defun org-ql--readme-predicate-list () "Return an Org list string documenting predicates." (concat (unpackaged/docstring-to-org "Arguments are listed next to predicate names, where applicable.
Note that, for convenience, standard numeric comparator function symbols (
=', etc.) do not need to be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.\n\n") (s-join "\n" (->> org-ql-predicates (--sort (string< (symbol-name (plist-get it :name)) (symbol-name (plist-get other :name)))) (--map (-let* (((&plist :name name :docstring docstring :fn fn :args args) it) (args (->> args (--replace-where (listp it) (car it)) (--replace-where (eq '&rest it) '&optional)))) (if docstring (progn (setq docstring (s-replace "\n" " " docstring)) (format "+ ~%s%s~ :: %s" name (if args (format " %s" args) "") (unpackaged/docstring-to-org docstring))) (when (s-prefix? "org-ql-" (symbol-name name)) (warn "No docstring for: %s" name)) nil))) -non-nil))))
*** TODO Use async
If ~org-ql~ is loaded byte-compiled, the argument lists are not named properly (not sure why, as ~help-function-arglist~ is supposed to handle that). We could run the function in another Emacs process with ~async~ to avoid this.
** Info export options
** File-local variables