Compress prompts to speed up language model inference