|
1 | 1 | # Notes & Research |
2 | 2 |
|
3 | | -I made a whole 'blog' system for this... yet here we are.... |
| 3 | +These are my random thoughts or research on AI. |
4 | 4 |
|
5 | | -No, I'm not using ai to speak for me here. |
| 5 | +Tap the Entry Title above to open the Blog Entry List. |
6 | 6 |
|
7 | | - These are my thoughts, |
8 | | - |
9 | | - How ever scattered they may be. |
10 | | - |
11 | | -If you couldn't tell, I'm training my AIs on my own works. |
12 | | - |
13 | | - A personally made AI trained on personally made images / videos / photos / code / writing. |
14 | | - |
15 | | - That means I can copyright my generations, right? |
16 | | - |
17 | | - If I made every aspect of the AI & training data? |
18 | | - |
19 | | - - February 2025 |
20 | | - |
21 | | -I've begun on the core of the AI, as of May 24th, 2025. |
22 | | - |
23 | | - I have the beginnings of a 'Micro-Term' memory implemented to act as a gated-attention during inference. |
24 | | - |
25 | | -This, paired with automatic graph edge splitting ('Dynamic' in DGNN or DGAT) and use of geometric clustering, seems to be giving me values of a "remembered" object when it's outside of the dataset. |
26 | | - |
27 | | - Hopefully leading to bodily awareness of limbs, objects outside of the field of view, and other 'long term' tensors/classifications at a temporary scale. |
28 | | - |
29 | | -It's a 4d kernel, in that it uses an ESN to train on it's own mistakes, |
30 | | - |
31 | | - Basing it's decisions on prior back-propagation states/adjustments. |
32 | | - |
33 | | - The beginnings of a meta-learning process, I hope! |
34 | | - |
35 | | -I'm using a method I'm calling 'Stack Crunching', |
36 | | - |
37 | | - Where I agregate the time dependent weights into a "checkpoint" of sorts. |
38 | | - |
39 | | - This allows the ESN to have a 'baseline' understanding of data that I can parse into with vectors calculated from tensor weights found within a quantized version of the input data. |
40 | | - |
41 | | -You can assume that the 'ESN' is not a standard 'Echo State Network' anymore. |
42 | | - - May 2025 |
43 | | - |
44 | | - With a bit more research into the types of minds that brought us DeepMind, and their work on GNN networks, |
45 | | - |
46 | | - I read a bit of Petar Velickovic's work on topological deep learning and the geometry of GNNs. |
47 | | - |
48 | | - Coming to find out that my idea of 'Stack Crunching' is similar to 'Squashing' in GNNs. |
49 | | - |
50 | | - So I've been inspired to propperly name my neural structure- |
51 | | - |
52 | | -It's a Dynamic Pointer-Attention Message Passing Neural Network with Affine-Projections |
53 | | - |
54 | | - or a dPA-MPNN |
55 | | - |
56 | | - But I must say, this isn't Affine Projections like in the papers, |
57 | | - |
58 | | - It's more like a 'projection' of the data into 'pointer' space; |
59 | | - |
60 | | - Actual Affine Matricies. |
61 | | - |
62 | | - I am a Technical Artist first before an AI Researcher after all, BOIDS! |
| 7 | +Or Tap the Left & Right arrow buttons to change Entry. |
63 | 8 |
|
64 | | -It all comes down to BOOOOIIIIIDDDDSSSSSS instead of Adam, baby! |
65 | | - |
66 | | -Because, what is Adam? It's a direction to move in a field of numbers, with momentum and a learning rate. |
67 | | - |
68 | | - Yet... That's just a simple Boid, now isn't it? |
69 | | - |
70 | | - Just without a few of the more advanced rules, which make boids feel so alive! |
71 | | - |
72 | | -Having some Tiny Brains running around in hyperdimensional space like little buggers running around avoiding each other. |
73 | | - |
74 | | - Because if they collide, double activation happens when it may not be desired. |
75 | | - |
76 | | - (I'm happy I finally saw a paper on Tiny Brains, giving some of my ideas credence, cause it fits! .. in my mind.) |
77 | | - |
78 | | -(Only difference is that it was a [study](https://www.nature.com/articles/s41586-025-09142-4) into small biological systems, not artificial ones... but I'm gettin there!) |
79 | | - |
80 | | -Update : At the time, I had been seeing some papers talking about "Tiny Brains" being used in AI, hence the term. |
81 | | - |
82 | | - But this biological study really helped solidify some ethereal concepts. |
83 | | - |
84 | | - See August 2nd 2025 for some thoughts. |
85 | | - |
86 | | - - July 2025, Updated August 2025 |
87 | | - |
88 | | -I'd like to believe I'm moving in the right direction with the feedback systems I'm developing. |
89 | | - |
90 | | - But been further creating other architectures to see how they operate. |
91 | | - |
92 | | -I created a GAN for upressing, which helped me understand a bit better the pairing of mental structures between both our brain's hemispheres. |
93 | | - |
94 | | - So I added a time based memory to check if the training was moving in the right direction. |
95 | | - |
96 | | - It definitely helped guide training a bit quicker. |
97 | | - |
98 | | -Shows my knowledge base that I'm impressed by back-up supported learning... |
99 | | - |
100 | | - But'is proof of concept! |
101 | | - |
102 | | -Adversarial networks exist in nature to guide a 'single' thought's path. |
103 | | - |
104 | | - Yet in the case of Group Think between humans, |
105 | | - |
106 | | - Balance is never reached. |
107 | | - - August 1st 2025 |
108 | | - |
109 | | -I've been looking into neural bundles in the brain. There is an implicit "delay" in the flow of information that I'm interested in. |
110 | | - |
111 | | - As signals move between neurons, some connections take a longer path than others to get to the same destination. |
112 | | - |
113 | | - For as much as I interpreted it. |
114 | | - |
115 | | -There is 6 main layers of neurons in the cerebral cortex, |
116 | | - |
117 | | - Of these, the 4th seems to allow for delays in processing. |
118 | | - |
119 | | - The 5th layer then introduces a dense layer of pathways for the signals to travel through. |
| 9 | +And no, I'm not using ai to speak for me here. |
120 | 10 |
|
121 | | - This is where I think another form of delay is introduced. |
122 | | - |
123 | | -I was comparing Mice and Wallaby brains, |
124 | | - |
125 | | - While Mice are likely more intelligent, |
126 | | - |
127 | | - Wallabies have more connections with denser pathways, it seems. |
128 | | - |
129 | | -Wallabies have more glial cells within slices of the brain compared to Mice. |
130 | | - |
131 | | - But mice had more neurons in the same slices. |
132 | | - |
133 | | -I'd like to believe, this doesn't mean there is a "better" brain here. |
134 | | - |
135 | | - But rather, different types of brains that are suited for different tasks. |
136 | | - |
137 | | -Wallabies are known to be social animals when food is plentiful, |
138 | | - |
139 | | - Yet solitary when food is scarce. |
140 | | - |
141 | | -Mice are known to be social animals, |
142 | | - |
143 | | - And have shown empathy towards other mice in distress, |
144 | | - |
145 | | - And share food with other mice when they are in need. |
146 | | - |
147 | | -Why do I bring this up? |
148 | | - |
149 | | - I believe there is similar deductive reasoning, just at a different scale. |
150 | | - |
151 | | -Both Wallabies and Mice are making a choice based on the environment and situation, |
152 | | - |
153 | | - While considering the well-being of others, just in different ways. |
154 | | - |
155 | | -The delay in neural firing could be a factor in this. |
156 | | - |
157 | | - So I'd like to explore this in my own AI. |
158 | | - |
159 | | -We all know size of the brain can determine intelligence, |
160 | | - |
161 | | - But so does the structure of the brain. |
162 | | - |
163 | | - - August 2nd 2025 |
164 | | - |
165 | | -So, more'n more there are some rather choice words about AI online. |
166 | | - |
167 | | -I wanted to put my personal ai dev views on record somewhere, for those who care. |
168 | | - |
169 | | -I read the I Ching and it put life into a different perspective. |
170 | | - |
171 | | - Letting me down the path of researching Taoism |
172 | | - |
173 | | -As with many of the other religious texts I looked into, |
174 | | - |
175 | | - Amazing imagery was used to teach morals and help guide the lost, |
176 | | - |
177 | | - But organized religion as a whole feels a little off to me. |
178 | | - |
179 | | -I don't hold any particular belief or religion at this point. |
180 | | - |
181 | | - But I would use the cliche 'spiritual' to describe my outlook |
182 | | - |
183 | | -I then visited the Buddhist monastery in Carmel NY, |
184 | | - |
185 | | - Greeted by the largest buddha statue in north america. |
186 | | - |
187 | | - In awe of the multitudes of multitudes of hand-carved buddha statuettes in audience of the massive statue of buddha I pale in comparison before. |
188 | | - |
189 | | -I'd highly suggest visiting the monastery if you ever find yourself in the area! |
190 | | - |
191 | | -I think it was walking through the rows of 18 arahants statues, of those who reached nirvana, helped me realize, |
192 | | - |
193 | | - Religion is about teaching the lessons of god(s), |
194 | | - |
195 | | - Yet understanding balance is what's inside all of us as Humans, |
196 | | - |
197 | | - Just gotta find it! |
198 | | - |
199 | | -So, |
200 | | - |
201 | | - I'd like to hope I'm nuance-first with my approach to my ai development. |
202 | | - |
203 | | - I'd like to believe in an AI which can understand... |
204 | | - |
205 | | - That overlooked concepts matter in Health and Wellbeing. |
206 | | - |
207 | | -Realistically, the Buddhist Precepts feel like a good place to start for alignment. |
208 | | - |
209 | | - Even as people. |
210 | | - |
211 | | - Which is more than I can say for myself.... |
212 | | - |
213 | | - I'm a hedonist at times, absurdist the rest |
214 | | - |
215 | | - (Absolute terms are fun to use, hyperbole be a thing) |
216 | | - |
217 | | -I very much enjoyed working on family films, |
218 | | - |
219 | | - Seeing the fans in comments online, |
220 | | - |
221 | | - And wish to work on more animated features soon. |
222 | | - |
223 | | - I have hope in humanity |
224 | | - |
225 | | -May the few not ruin it for those of us trying to explore new horizons. |
226 | | - |
227 | | - - August 15th,16th 2025 |
| 11 | +These are my thoughts, how ever scattered they may be. |
0 commit comments