nightmedia commited on
Commit
7dfd663
·
verified ·
1 Parent(s): b166fc6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -3
README.md CHANGED
@@ -16,9 +16,7 @@ library_name: mlx
16
  Hi Spock!
17
  We are going to analyze the cognitive abilities of a few quantizations of this model
18
 
19
- The Deckard(qx) quants are in a mixed precision quantization:
20
- - qx64x has data at 4 bit, while the attention paths, head, and embeddings are at 6 bit
21
- - qx86x has data at 6 bit, while the attention paths, head, and embeddings are at 8 bit
22
 
23
  The Deckard formula was inspired from my Nikon Noct Z 58mm F/0.95 for its human-like rendering, sharp details, thin depth of field, and pattern-rich background blur that humans find pleasing. In interaction, these models have a specific character that associated the name, quite often reaching out to metaphors. I used this idea in the transformer layer design, by adding enhanced attention paths in high bit size every four layers, additionally to setting the heads and embeddings to high bit.
24
 
 
16
  Hi Spock!
17
  We are going to analyze the cognitive abilities of a few quantizations of this model
18
 
19
+ The Deckard(qx) quants are in a mixed precision quantization, with data at 6 bit, while the attention paths, head, and embeddings are at 8 bit
 
 
20
 
21
  The Deckard formula was inspired from my Nikon Noct Z 58mm F/0.95 for its human-like rendering, sharp details, thin depth of field, and pattern-rich background blur that humans find pleasing. In interaction, these models have a specific character that associated the name, quite often reaching out to metaphors. I used this idea in the transformer layer design, by adding enhanced attention paths in high bit size every four layers, additionally to setting the heads and embeddings to high bit.
22