Previously, the precedence of `as` binding was higher than the binary
operators. This results in unexpected behaviors, `1 + 2 as $x | -$x`
was `-1` instead of `-3`. The commit 92daac90f6 (PR #3053) changed
the behavior to emit syntax error on this case. Now, this commit allows
binary operators with `as` binding. This fixes#2446, and fixes#3284.
This commit fixes signed integer overflow and SEGV issues on growing
arrays and objects. The size of arrays and objects is now limited to
`536870912` (`0x20000000`). This fixes CVE-2024-23337 and fixes#3262.
This commit improves the parser and compiler error messages, making
it more easier to understand where in the filter the problem is.
Like modern compilers, jq will show the line in the filter where the
error occurred, will display squiggly line under the invalid token.
This commit improves the parser to fix some unintuitive results of the unary
operator `-` (e.g. `[-1 as $x | 1,$x]` is now `[1,-1]`, previously `[-1,-1]`).
Also, this commit now allows to use binary operator expressions, `reduce`,
`foreach`, `if`, and `try` expressions for object values.
This commit refactors `splits/2` implementation using `foreach` syntax
and reduces intermediate arrays. This refactoring removes an unnecessary
private (and undocumented) filter `_nwise` and closes#3148.
This commit drops support for parsing NaN with payload in JSON like
`NaN123` and fixes CVE-2024-53427. Other JSON extensions like `NaN` and
`Infinity` are still supported. Fixes#3023, fixes#3196, fixes#3246.
Using `calloc` function with a count of zero elements is implementation
defined and it may cause allocation errors on some platforms (ref #3277).
This commit fixes `jv_mem_calloc` callers to avoid undefined behavior.
When we use a large format string to `strftime/1` and `strflocaltime/1`,
we get segmentation fault. This happens because the result buffer is
allocated on the stack using `alloca` function, and `strftime` function
writes out of the space when the size is not enough to format time.
I fixed the segmentation fault issue by allocating the result buffer on
the heap using `malloc` function, because `alloca` function does not
provide a way to detect insufficient space.
This commit improves the performance of repeating strings, by copying
the result string instead of the string being repeated. Also it adds
an error message when the result string is too long.
Previously, the configure script automatically enables Valgrind during
testing when the valgrind command is found. However, running tests with
Valgrind is quite slow, so it is better to disable it by default, and
to enable it only when the user specifies the `--enable-valgrind` flag.
On OpenBSD, some prototypes are hidden behind `_BSD_SOURCE`.
Defining this variable fixes#3252, and also fixes#3276.
This can be seen as a regression of #2631.
Previously, `unique` and `unique_by` filters are implemented using
`group_by` and then `map(.[0])`. This commit re-implements in C, to
avoid unnecessary boxing of grouping, and improves the performance.
Whether `char` has a sign is implementation dependent, and `c >= 0` may
result in a compiler warning. We can use bitwise operator to check
whether the character is a control character, regardless of the sign.
The code of `jv_number_value` is intended to cache the double value of
literal numbers, but it does not work because it accepts the `jv` struct
by value. This patch fixes the behavior by checking if the double value
is `NaN`, which indicates the unconverted value. This patch improves the
performance of major use cases; e.g. `range(1000000)` runs 25% faster.
We no longer use the `--root` option of `build_website.py` since we have
moved to jqlang.org (from jqlang.github.io/jq where the root was `/jq`).
We don't need the option on local development either since `http.server`
has the `-d` option to specify the directory to serve from.
When the UPDATE query in a reduce or foreach syntax does not emit
a value, it should simply skip updating the state variable. But
currently it resets the state variable to null.
I also changed the behavior of limit/2 with negative count to emit an error, like nth/2 does.
Also, I redefined nth/2 using skip/2 to minimize the impact of growth of builtin filters.
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
* Always build docker image on CI
* Revert "Bump actions/download-artifact from 3 to 4"
This reverts commit 9e2695e117.
* Group Dependabot PRs for official actions
Although #2839 fixed the overflow of exponent subtraction,
there still is possibility of overflow in the `D2U` macro.
This patch fixes the overflow in the `D2U` macro, and also
truncates the maximum digits to `DEC_MAX_DIGITS`.
The decNumber library subtracts the exponents of two numbers,
we make sure to limit the number of digits not to make it overflows.
Since the maximum adjusted exponent is `emax` and the minimum is
`emin - digits + 1`, we follow `emax - (emin - digits + 1) <= INT32_MAX`.
Previously constant folding of zero division (e.x. 1/0) produces a
compile error. This was incorrectly implemented by checking if the
division result is infinite, so produces wrong results compared to the
query where no constant folding is processed (e.x. 1e308/0.1). This
patch delays the operation when the divisor is zero. This makes the
results more consistent, but changes the exit code on zero division from
3 to 5. Also 0/0 now produces the zero division error, not NaN.
This patch also fixes the modulo operation. Previously constant folding
logic does not take care of the % operator, but now it folds if the both
dividend and divisor are safe numbers to cast to the integer type, and
the divisor is not zero. This patch also fixes some code that relies on
undefined cast behaviors in C. The modulo operation produces NaN if
either the dividend or divisor is NaN.
* Bump up Bootstrap to v5.3.1, Bootstrap Icon to v1.10.5.
* Use autoComplete.js to drop dependency on jQuery and typeahead.js.
* Support dark mode.
* New svg logo and icon with responsive color mode support.
* Normalize section ids to lower kebab-case for easiness of linking.
* Use relative paths for links for local development (--root /output).
* Various markup cleanups and accessibility improvements.
The option naming --nul-output was confusing, especially when we have a
similar option for input stream in the future (--nul-input vs --null-input).
Based on the observation of other command line tools, we rename the option
to --raw-output0. We also drop the short option -0 to avoid confusion on
introducing the NUL-delimited input option.
Unlike the other command line tools outputting file names with NUL delimiter,
jq deals with JSON, and its strings may contain NUL character. To protect
users from the risk of injection attacks, we abort the program and print an
error message before outputting strings including NUL character. Closes#2683.
- Add error/0 and mentions null input behavior (close#2231)
- Explain value iterator suffix syntax .foo[] (close#1047)
- Mention array slicing is also zero-based (close#2094)
- Add examples of input and inputs filters (close#2216, close#2470)
- Improve sort_by about multiple values (close#2103, close#2467, close#2474)
- Improve foreach section and simplify examples (close#1148, close#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close#2036, close#2412)
- Add non-string examples of index/1, rindex/1 (close#1422)
- Simplify the example of truncate_stream/1 (close#1736)
itchyny