28 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Kye Gomez	537b116b3e	just use adam for now in training maybe add muon later	2026-04-19 23:34:58 -04:00
Kye Gomez	137cd8832e	readme	2026-04-19 23:21:50 -04:00
Kye Gomez	f37d405a81	readme	2026-04-19 23:21:39 -04:00
Kye Gomez	ef2cac6c3a	readme	2026-04-19 23:20:53 -04:00
Kye Gomez	12f6c5b32e	[docs][readme-badges][add pypi twitter github badges to ... readme][improvement][init-sort][sort imports alphabetically in init][improvement][version-bump][bump version to 0.3.0][feat][tokenizer-class][add MythosTokenizer to init exports][feat][test-tokenizer][add tokenizer test suite with printed output]	2026-04-19 23:18:05 -04:00
Kye Gomez	5ffb897dcf	[feat][training-script][add 3b fineweb-edu training ... script][feat][tokenizer][add MythosTokenizer class with encode decode][improvement][deps][add transformers and datasets dependencies][docs][readme-training][add training section with run commands][improvement][pyproject][pin torch and add new deps]	2026-04-19 22:48:30 -04:00
Kye Gomez	97bc414977	use gpt-oss tokenizer because it's a great tokenizer	2026-04-19 22:28:09 -04:00
Kye Gomez	5cfef742b5	[feat][dropout][add dropout to config attn and ... residuals][docs][datasets][add recommended training datasets doc][docs][readme-datasets][link datasets doc in readme]	2026-04-19 22:15:41 -04:00
Kye Gomez	0623ceb960	Update README.md	2026-04-19 22:01:10 -04:00
Kye Gomez	a825ba217f	[Update variants with parameter count equation]	2026-04-19 21:55:06 -04:00
Kye Gomez	f00d10b59f	more references fix	2026-04-19 21:39:13 -04:00
Kye Gomez	e522ecf630	clean up	2026-04-19 15:04:30 -04:00
Kye Gomez	1c54259fa8	format	2026-04-19 15:04:03 -04:00
Kye Gomez	9cce1c1401	variants	2026-04-19 15:03:45 -04:00
Kye Gomez	299a0cdb0a	pip and readme	2026-04-19 08:33:09 -04:00
Kye Gomez	806a8da1d6	[bugf][lora-b-init][fix zero-init B making adapter always output zero][bugf][lti-get-a][fix 0 times ... inf NaN in log space computation][improvement][rope-theta-test][exclude degenerate dim0 from theta angle comparison]	2026-04-18 23:51:01 -04:00
Kye Gomez	53f786afda	disclaimer and rope tests	2026-04-18 23:44:31 -04:00
Kye Gomez	14a806c2a1	installation	2026-04-18 20:48:34 -04:00
Kye Gomez	7abd3e5d20	cleanup	2026-04-18 20:44:23 -04:00
Kye Gomez	0818bf815b	readme	2026-04-18 20:43:05 -04:00
Kye Gomez	fb2939b6c4	[DOCS] [LICENSE]	2026-04-18 20:20:47 -04:00
Kye Gomez	0699c00c94	tests	2026-04-18 10:12:45 -04:00
Kye Gomez	c258cdc8da	example .py	2026-04-18 09:55:56 -04:00
Kye Gomez	791626667d	[FEAT][Main.py]	2026-04-18 09:53:06 -04:00
Kye Gomez	4ce503f3dd	moe	2026-04-18 09:15:46 -04:00
Kye Gomez	79b916d999	paper	2026-04-18 09:07:19 -04:00
Kye Gomez	4aea9d9dd5	reference	2026-04-18 09:02:50 -04:00
Kye Gomez	8fa70224c7	Initial commit	2026-04-18 08:57:52 -04:00