I have to admit that I don't know how to use AI but have a project that's AIish.

Try https://claude.ai/, it's great for code. It can analyse your code and then you can ask it questions about how to improve specific things - you just ask questions like you would ask a person.

ChatGPT or even Copilot can be very helpful too.

Good news: Earlier today, I gained .7% compressibility in my text adventure's text and decreased its total main parts' size from 8.32k down to 8.27k. 😀 That's not much but is something.

Down to 8.25k due to tweaking and optimizing. 😀 I had 8.24k but don't know what happened to it. 🙁

Harry Potter wrote on 2026-03-07, 15:31:

but don't know what happened to it.

Are you using source control? Make a note of the performance with each change you check in and then you'll never be wondering what happened.

I'm sorry about this, but what's "source control"?

Git, Subversion, CVS, etc.

Keeping a log of the changes of your source and documentation and allow you to compare, merge, rollback different versions of your source tree as well as individual files.

If you're not using source control then you really should be.

I do back up my works but often not diligently enough. 🙁

Earlier today, I found where the numbers went: they went to a different version of the C compiler I'm using. 😀

Earlier today, I managed to decrease the size of the test code but only slightly. I have no idea what I'm doing wrong, but my best compression technique only approaches 50%. 🙁 Maybe it's with the way I'm handling tokens.

Still little progress, but tokenization is doing a little better. Here's the current code for tokenization:

1unsigned parsetokens1main (void)
2{
3	int i, j, k, l, m, o;
4	unsigned curstrlen, curtoklen;
5	unsigned newtok;
6	int n;
7	int bestcurtok, bestcurtoklen;
8	//struct tok2buf * besttok[16];
9	struct string_buffer *teststr;
10	char *besttokptr;
11	char* tokstr;
12	unsigned toklen;
13	curstr=strings;
14	while (curstr) {
15		curstrlen=curstr->inlen-1;
16		for (i=0; i<curstrlen;) {
17			bestcurtok=-1; newtok=1; bestcurtoklen=3;m=0;
18			for (j=i-1; j>=0; --j) {
19				l=getlenmatch(&curstr->in[i], &curstr->in[j]);
20				if (l==bestcurtoklen) {
21					newtok=0;
22					bestcurtok=j;
23					bestcurtoklen=l;
24				} else if (l>bestcurtoklen) {
25					newtok=1; m=1;
26					bestcurtok=j;
27					bestcurtoklen=l;
28				}
29			} if (curstr->prev) {
30				teststr=curstr;
31				while (teststr=teststr->prev) {
32					if (teststr->inlen>=3) {
33						for (k=teststr->inlen-3; k>=0; k--) {
34							l=getlenmatch(&curstr->in[i], &teststr->in[k]);
35							if (l==bestcurtoklen) {
36								newtok=0;
37								bestcurtok=j;
38								bestcurtoklen=l;
39							} else if (l>bestcurtoklen) {
40								newtok=1; m=1;
41								bestcurtok=j;
42								bestcurtoklen=l;
43							}
44						}
45					}
46				}
47			}
48			newtok=1;
49			for (j=0; j<tok2bufsize; j++) {
50				if (bestcurtoklen==tok2buf[j].len && !memcmp(&curstr->in[i], tok2buf[j].token, bestcurtoklen)) {
51					newtok=0; break;
52				}
53			}
54			if (newtok && bestcurtoklen>=4) {
55				addtotok1 (&curstr->in[i], bestcurtoklen);
56			} else {
57				tok2buf[j].occur++;
58			}
59			if (bestcurtoklen>=4) i+=bestcurtoklen;
60			else i++;

61		}
62		curstr=curstr->next;
63	}
64	sorttokens1();
65	compresstoks();
66	compresstoks2();
67	curstring=strings;
68	while (curstring) {
69		complit_5a();
70		curstring=curstring->next;
71	}		
72	writetokstofile();
73	writestrs();
74	writedeclstrs();
75	return 0;
76}
77
78void sorttokens1 (void)
79{
80	unsigned i, j, k, l;
81	struct tok2buf tmpswaptok;
82	for (i=0; i<tok2bufsize; i++) {
83		tok2buf[i].saved=((tok2buf[i].len)*(tok2buf[i].occur));
84		if (tok2buf[i].occur<3) tok2buf[i].saved=0;
85	}
86	for (i=0; i<tok2bufsize-1; i++) {
87		k=i;
88		for (j=i; j<tok2bufsize; j++) if (tok2buf[j].saved>tok2buf[k].saved) k=j;
89		if (k!=i) {
90			memcpy (&tmpswaptok, &tok2buf[i], sizeof(tmpswaptok));
91			memcpy (&tok2buf[i], &tok2buf[k], sizeof(tmpswaptok));
92			memcpy (&tok2buf[k], &tmpswaptok, sizeof(tmpswaptok));
93		}
94	} 
95	for (i=0; i<tok2bufsize; i++) {
96		if (tok2buf[i].saved<20) {tok2bufsize=i; break;}
97	}
98	collecttokens();
99}

I was wrong: I implemented the attached code, and it cost me some compressibility. 🙁 I recently managed to buy some points in one version of PrintTok2, though. 😀 Any other ideas I can implement?

Hi! I was recently told on another forum that PrintTok2 was actually much better than I thought: until then, I didn't have an accurate comparison. However, one version is not doing as well as I'd like. 🙁 I recently gained 25 bytes there by shortening a token comparison from the size of the token to the four bytes. The compressed size in the experiment is now 2,863 bytes, but I want that number to be down to 2,850 bytes or better. Other than tokenization, I'm using BPE and a form of Static Huffman Codes on lower-case letters and certain punctuation. I'm using LZ77 on another version of PrintTok2 but deprecated it here, as it would require way more code and data to decompress than I gained via better compression.

BTW, I forgot to mention that the text adventure is now down to 7.92k. 😀