Commit Graph
406 Commits
Author SHA1 Message Date
Mattias Wadman 16ae275d4c Fix leaks on invalid input for strftime, strflocaltime and mktime 2023-07-24 16:45:43 +02:00
David Korczynski 97c6d2848c Add fuzzer targeting load_file
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-24 15:45:12 +02:00
David Korczynski cf3c11bde8 Add compile fuzzer for OSS-Fuzz
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-24 13:44:27 +02:00
itchyny ed334b536f Improve manual in various ways (inputs, sort_by, foreach sections, etc.) (#2744)
- Add error/0 and mentions null input behavior (close #2231)
- Explain value iterator suffix syntax .foo[] (close #1047)
- Mention array slicing is also zero-based (close #2094)
- Add examples of input and inputs filters (close #2216, close #2470)
- Improve sort_by about multiple values (close #2103, close #2467, close #2474)
- Improve foreach section and simplify examples (close #1148, close #2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close #2036, close #2412)
- Add non-string examples of index/1, rindex/1 (close #1422)
- Simplify the example of truncate_stream/1 (close #1736)
2023-07-24 20:24:44 +09:00
Nico Williams 1cf6515c06 Fix try/catch catches more than it should #1859
Close #1885, #2140, #2011, #2220, #2485, #2073

Rename the FORK_OPT opcode to TRY_BEGIN, add a TRY_END opcode, and wrap
errors when raising through a TRY_END so that they will not be caught by
the matching TRY_BEGIN.

Now a `try exp catch handler` expression generates code like:

    TRY_BEGIN handler
    <exp>
    TRY_END
    JUMP past_handler
    handler: <handler>
    past_handler:
    ...

On backtrack through TRY_BEGIN it just backtracks.

If anything past the whole thing raises when <exp> produced a value,
then the TRY_END will catch the error, wrap it in another, and
backtrack.  The TRY_BEGIN will see a wrapped error and then it will
unwrap and re-raise the error.

If <exp> raises, then TRY_BEGIN will catch the error and jump to the
handler, but the TRY_BEGIN will not stack_save() in that case, so on
raise/backtrack the TRY_BEGIN will not execute again (nor will the
TRY_END).
2023-07-24 13:20:21 +02:00
itchyny 98f709d0c1 Fix stderr to output string with no decoration (fix #2063) 2023-07-23 17:06:00 -05:00
Nicolas Williams f64ef1a7d7 Fix --stream-errors w/o --stream, and doc+test (close #2748) 2023-07-23 03:33:28 -05:00
itchyny c9c45d7f03 Add tests for --arg, --argjson, and similar options 2023-07-22 12:29:24 -05:00
itchyny 99a77f724b Add a regression test for #2000 2023-07-21 09:01:28 +09:00
pkoppstein 3ec66c858c manual.yml: revise section on identity, clarifying what is subject to change
Mostly clarifications w.r.t. numbers.

The anomalous behavior of leading minus signs is documented, with an explicit notice that this will probably change.
2023-07-19 22:32:24 -05:00
itchyny 0615d4982c Respect NO_COLOR environment variable to disable color output 2023-07-19 00:05:41 -05:00
0b558f6ed4 Make object key color configurable (close #1739, #1791, #2638)
Co-authored-by: David Haguenauer <ml@kurokatta.org>
Co-authored-by: Eric Pruitt <eric.pruitt@gmail.com>
2023-07-18 17:53:29 -05:00
Emanuele Torre 15fd31de78 Define examples in the "examples" field instead of "example"
Only examples defined in the "examples" field are generated, if they are
defined in "example", they are just ignored.

Also fix a bunch of incorrect tests.
2023-07-18 12:12:50 -05:00
Emanuele Torre 01dfd8b86d Populate captures also for zero-width matches
Instead of just using {"captures":[]}.

sub functions are use captures for replacement expressions.
If we don't populate, captures for empty matches, the replacement
expression is run with an empty object as input instead of an object
containing the named captures with "" as value:

* before:

  $ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
  "ZfooZbarZnull"

* after:

  $ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
  "ZfooZbarZ"

---

I also removed a redundant

  result = NULL;
  if (result) {
    ...
  }
2023-07-18 12:12:50 -05:00
Emanuele Torre bb0f898c01 Initialise jq_state->debug_cb{,_data} to NULL
To make debug/0 not call an uninitialised function pointer when using
--run-tests or when using a jq_state on which jq_set_debug_cb() has not
been called.
2023-07-18 12:12:50 -05:00
itchyny 9e4c71558a Split man.test to make tests pass without oniguruma (#2722) 2023-07-18 08:17:21 +09:00
Emanuele Torre 4b3090a9ab Make stream parser error when a non-empty object or array is used as key
Fixes #2463
2023-07-17 13:15:55 -05:00
Emanuele Torre e79335e3a5 Initialise jq_state->input_cb{,_data} to NULL in jq_init()
To avoid causing segmentation faults when  input/1  is called in a
jq_state on which  jq_set_input_cb()  has not been called; e.g. the one
used by  jq --run-tests.

That segfault could also be fixed in run_jq_tests() by calling:

  jq_set_input_cb(jq, NULL, NULL);

But I think it makes sense to just make jq_init() initialise those
values to NULL.

Ref: https://github.com/jqlang/jq/pull/2717#discussion_r1264338841
2023-07-16 00:03:46 -05:00
Emanuele Torre 1915b689f7 Fix indices/1 and rindex/1 in case of overlapping matches in strings
Fixes #2433
2023-07-14 23:40:37 -05:00
Emanuele Torre 4b1ac7c95f Parse nan in JSON as NaN instead of triggering a parse error (#2712)
* Fix memory leak for has(nan)

jv_array_get() used to be responsible of freeing the input array, but
since b5c4c3d67d, it is no longer called
if the key is nan. We need to free it manually to avoid leaking the
array.

* Parse  nan  in JSON as NaN instead of triggering a parse error

Fixes #2021
2023-07-14 07:12:27 +09:00
itchyny 092fef740a Make sure distcheck passes and run dist job on Pull Request 2023-07-13 10:20:04 -05:00
itchyny 689a4e660b Fix uri format to follow RFC 3986 (#2157) 2023-07-13 23:23:23 +09:00
itchyny b5c4c3d67d Fix has(nan) on arrays to output false 2023-07-12 09:40:05 -05:00
Emanuele Torre c99981c5b2 Update "tests/man.test" and remove superflous whitespace in manual.yml
tests/man.test was generated incorrectly and had some duplicate tests,
the new CI workflow didn't like that.
2023-07-11 00:24:50 -05:00
Nicolas Williams 1ec8be1422 Update tests/man.test 2023-07-10 14:18:27 -05:00
davkor 3df8f90c4e Add first fuzzer for integration with OSS-Fuzz.
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-10 13:57:38 -05:00
Emanuele Torre 6ca7973d49 Allow using { $__loc__ } as shorthand for { __loc__: $__loc__ }
n.b.: { $__loc__: "foo" }  is still not allowed.

$__loc__ cannot be overwritten, and it is always an object.
Using { $x: "foo" } where $x is not a string is an error, so there is
not much of a point in supporting that.
2023-07-10 12:04:54 -05:00
Emanuele Torre 193f432e08 Lex $foo as a single token, instead of using '$' IDENT
Previously, bindings were parsed as the combination of two tokens:
  '$' IDENT

This meant that using a keyword as variable name (e.g. $then, $label)
did not work.
Attempts were made to allow $keyword to work by adding some '$' Keyword
rules in the parser, but this did not allow $keyword in all places:

  jq --arg label foo -n '$label'  # ok

  jq -n '"foo" as $label | .'     # error

Treating $foo as a single token is much simpler, in my opinion.

This patch also changes how LOC is lexed: "$__loc__" instead of as
"__loc__" that gets combined with '$' in the parser.

This patch also disallows having spaces after '$' when recalling a
variable  `$ foo'  since that was probably just an unintentional side
effect of the implementation, and it was not documented.

Fixes #2675
2023-07-09 17:23:19 -05:00
itchyny 600e602548 Fix empty regular expression matches (fix #2565) 2023-07-09 12:51:53 -05:00
itchyny cac216a39c Fix manual on paths/1 to use boolean filter for its argument 2023-07-09 10:54:13 -05:00
Mattias Wadman ab1a68144d Support .a.[] and .a.[]? each syntax
Fixes #1699
2023-07-09 10:08:31 -05:00
itchyny d8072564c2 Fix nth/2 to emit empty on index out of range 2023-07-08 23:13:15 -05:00
itchyny cac3ea3726 Fix string multiplication by 0 (and less than 1) to emit empty string 2023-07-08 22:25:22 -05:00
itchyny 88b45771bd Fix string multiplication with a value between 0.0 and 1.0
This commit fixes a regression of 6306ac8966.
2023-07-08 22:25:22 -05:00
itchyny 371abc73ba Fix deletion using assigning empty against arrays (fix #2051) 2023-07-07 20:11:10 -05:00
itchyny 0ecededefa Remove deprecated filters: leaf_paths, recurse_down 2023-07-07 10:53:40 -05:00
itchyny 6944d81bc8 Fix dumping large floating numbers (fix #2367) (#2661) 2023-07-06 21:44:17 +09:00
pkoppstein 416e8ada7d builtin: add pick(stream)
pick(stream) works on both arrays and objects

Restrictions on the dot-paths are specified in manual.yml

See #2578
2023-07-05 23:48:31 -05:00
itchyny a881532b29 Fix exit code on JSON parse error (#2654) 2023-07-06 10:01:59 +09:00
Mattias Wadman 95625c3fdb Validate JSON for --jsonarg
Fixes #2572
2023-07-05 16:30:14 -05:00
itchyny 886a9b18b1 Make 0 divided by 0 results in NaN consistently (#2253) 2023-07-05 04:28:08 +09:00
Mattias Wadman dfd440f0de Setup path correctly after stream parsing a key:value object pair (#2642) 2023-07-05 04:23:52 +09:00
Mattias Wadman 12ce4e3b54 Support optional else-branch for if-then-elif-end (#2598) 2023-07-04 07:51:29 +09:00
itchyny 4dc2a2fb9d Implement scan/2 filter (#1961) 2023-07-04 07:48:29 +09:00
pkoppstein 83f375cc83 Revamp sub/3 to resolve most issues with gsub (and sub with "g") (#2641)
The primary purpose of this commit is to rectify most problems with
`gsub` (and also `sub` with the `g` option), in particular fix #1425 ('\b'),
fix #2354 (lookahead), and fix #2532 (regex == `"^(?!cd ).*$|^cd "`).

This commit also partly resolves #2148 and resolves #1206 in that
`gsub` no longer loops infinitely; however, because the new `gsub`
depends critically on `match/2`, the behavior when regex == `""` is
sometimes non-standard.

The documentation has been updated to reflect the fact that `sub`
and `gsub` are intended to be regular in the second argument.

Also, `_nwise/1` has been tweaked to take advantage of TCO.
2023-07-04 07:46:29 +09:00
pkoppstein 3c76a5b3f3 manual.yml: fix references to javascript, and clarify semantics of == (#2645)
remove inaccurate and confusing references to javascript; clarify semantics of ==
2023-07-03 12:35:41 +09:00
itchyny f88c4e5888 Move an example to the manual and regenerate man.test (#2639) 2023-06-30 07:45:29 +09:00
itchyny 6d694a4a58 Validate module metadata to be an object (#2357) 2023-06-28 17:59:46 +09:00
itchyny 98835eec24 Fix halt_error to print message without prefix in raw mode (fix #1902) (#2632) 2023-06-27 07:33:21 +09:00
itchyny 73966c80ec Fix number lexer to avoid conflict with object indexing (#2254) 2023-06-27 07:24:49 +09:00