Fix the first line of UTF-16BE input swallowing the second line

In UTF-16BE a newline is 00 0A, so the initial read_until(b'\n') usually
already ends exactly at the end of line 1. The BE branch then always called
read_utf16_line, which read on to the next newline, merging lines 1 and 2.
Only keep reading when the first read did not end on a complete 00 0A.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VVn4SeAXPsUajXT2rtHQfo
This commit is contained in:
Claude committed 2026-10-16 19:50:14 +00:00
1 parent 62b439dae3
commit c2adbf6caa
3 files changed
+20 -1

No files matched your search

+13
View File
@@ -2101,6 +2101,19 @@ fn utf16be() {
.stdout(" 1 上一伊刀\n 2 foo bar\n 3 hello world\n");
}
#[test]
fn utf16be_first_line_not_merged_with_second() {
// The first line of a UTF-16BE file must not swallow the second line
bat()
.arg("--decorations=always")
.arg("--style=numbers")
.arg("--color=never")
.arg("test_UTF-16BE.txt")
.assert()
.success()
.stdout(" 1 hello world\n 2 this is a test\n");
}
// Regression test for https://github.com/sharkdp/bat/issues/1922
#[test]
fn bom_not_stripped_in_loop_through_mode() {