vault backup: 2024-06-10 - 1 files

Affected files:
Monthly Notes/Jun 2024 notes.md

Monthly Notes/Jun 2024 notes.md

@@ -0,0 +1,15 @@
+
+## notable reads
+
+- [A Picture is Worth 170 Tokens: How Does GPT-4o Encode Images?](https://www.oranlooney.com/post/gpt-cnn/)
+- sigma-gpt https://news.ycombinator.com/item?id=40608413
+	- The authors randomly permute (i.e., shuffle) input tokens in training and add two positional encodings to each token: one with the token's position and another with the position of the token to be predicted. Otherwise, the model is a standard autoregressive GPT. The consequences of this seemingly "simple" modification are significant:
+	- The authors can prompt the trained model with part of a sequence and then decode the missing tokens, all at once, in parallel, regardless of order -- i.e., the model can in-fill in parallel.
+	- The authors can compute conditional probability densities for every missing token in a sequence, again in parallel, i.e., densities for all missing tokens at once.
+	- The authors propose a rejection-sampling method for generating in-fill tokens, again in parallel. Their method seems to work well in practice.
+
+## discussions
+
+
+- forcing AI on to us
+	- msft recall default https://news.ycombinator.com/item?id=40610435