Details, Fiction and mamba paper
Jamba is a novel architecture constructed on a hybrid transformer and mamba SSM architecture produced by AI21 Labs with fifty two billion parameters, rendering it the biggest Mamba-variant made so far. it's got a context window of 256k tokens.[twelve] We evaluate the effectiveness of Famba-V on CIFAR-100. Our final results clearly show that Famba-