JPEG compression levels used in API calls can change the ranking lineup of models on visually prompted benchmarks.
Sources
- Visually Prompted Benchmarks Are Surprisingly Fragile (arxiv.org)
- https://www.researchgate.net/publication/398937366_Visually_Prompted_Benchmarks_Are_Surprisingly_Fragile (researchgate.net)
- https://lisadunlap.github.io/vpbench/ (lisadunlap.github.io)
- https://github.com/TonyLianLong/VPBench (github.com)
More in Technology
- A 100-kilowatt lunar nuclear reactor targeted by NASA is described as 10,000 times less powerful than a typical nuclear power plant.
- In NEJM Image Challenge cases where fewer than 40% of physicians selected the correct diagnosis, Claude 3.7 achieved 86.5% accuracy and GPT-4o achieved 78.8% accuracy.
- Researchers used artificial intelligence and X-ray scans to decipher parts of a 2,000-year-old papyrus scroll burned in the A.D. 79 eruption of Mount Vesuvius.
- The paper 'The unusual suspects' by N. Fatma, M. Chinnakotla, M. Shrivastava, S. Singh, and S. Markovitch appeared in the Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence on 4 February 2017, spanning pages 1107–1113.
- Under the draft EU AI Regulation, conformance with harmonised standards creates a presumption of conformity for high-risk AI applications and services.
- Slide and transparency series and floppy disks were among publication formats once collected by the Deutsche Nationalbibliothek that have largely disappeared from the market.