feat: add comment support to saccharine language #25
Reference in New Issue
Block a user
Delete Branch "feat/comment-support"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Description
The saccharine language previously lacked comment support, preventing proper code documentation.
This PR implements '#' comment syntax similar to Python.
Comments can appear on their own line or at the end of a line, with all content after '#' ignored until the next newline or EOF.
The tokenizer now detects '#' and skips characters appropriately without creating tokens.
Decisions
Comments are silently consumed during tokenization rather than being preserved as tokens, keeping the token stream clean for the parser.
The implementation preserves newlines after comments by using the iterator's Back() method, allowing them to be processed as soft breaks.
Benefits
Developers can now document their saccharine code with inline and full-line comments.
The implementation is minimal and efficient, adding no overhead to the token stream.
Tests verify that comments work correctly in various positions without breaking code execution.
Checklist
<type>/<description>). Always use underscores.Closes #24