Commit Graph

3 Commits

Author SHA1 Message Date
x a32b7e4693 Use TokenCounter and correct unsafe pointer usage
Record write errors in TokenCounter.Printf and add Println method
to propagate write failures. Route file-read warnings to stderr
instead of mixing them into token output. Use unsafe.Add with a
base pointer when computing the drop target on Windows.
2026-01-31 03:33:36 +01:00
x 14dfacea29 Refine TokenCounter handling and output
Decode across Write calls and buffer partial runes. Approximate
tokenization by splitting long alphanumeric runs (>4) and properly count
spaces and punctuation. Remove TokenCounter Printf/Println helpers and
update callers to use fmt.Fprintf/Fprintln. Avoid writing when the
underlying writer is nil.
2026-01-31 03:24:17 +01:00
x 7624928d99 init ctx 2026-01-30 19:21:54 +01:00