Derek Arnold

Visual language models show widespread visual deficits on neuropsychological tests - Nature Machine Intelligence Tangtartharakul and Storrs use standardized neuropsychological tests to compare human visual abilities with those of visual language models (VLMs). They report that while VLMs excel in high-level obje...

Our latest paper, “Visual language models show widespread visual deficits on neuropsychological tests”, is now out in Nature Machine Intelligence: www.nature.com/articles/s42...

Non-paywalled version:
arxiv.org/abs/2504.10786

Tweet thread below from first author @genetang.bsky.social...

No, but we probably should have done. Still can of course, but I doubt if it would work because we asked people for absolute ratings, as opposed to asking them to rate their own experiences relative to one another. That approach may be better for detecting within p variance

Now available without randomly placed repeated figures...

doi.org/10.1016/j.co...

Say him in Brisbane. Brilliant

Imagery modulates the pupillary response, but this does not reliably index differences in imagery vividness. Most people report that they can imagine seeing things in their mind’s eye. But there are large individual differences. A small proportion of people r…

New Paper: Pupillary responses are not a reliable index of differences in imagery vividness.

Our search for more reliable metrics of imagery continues...
www.sciencedirect.com/science/arti...

If you keep it short (an hour or less) and include coffee and cake, you can always can it if it does not work, but the cake should prove popular

Interested in other experiences.

I think it has a lot to do with how time poor you are. Individual meetings are usually better, but can get challenging depending on other demands on your time.

Lab meetings create opportunities for cross pollination, but are often unproductive virtue signaling

People report having consistent idiosyncratic ‘diets’ of imagined sensations when they re-experience the past, and pre-experience the future To some extent, humans can re-experience the sensations of past events and pre-experience the future. These capacities are inter-related. But there are substantial individual differences. At the extre...

New Preprint: People in general have idiosyncratic imagined experiences characterised by salience differences. Some have more salient imagined sensations of smell than of imagery, while most have the opposite - and these differences shape people's daily lives.
www.biorxiv.org/content/10.6...

Corollary Discharge Dysfunction to Inner Speech and its Relationship to Auditory Verbal Hallucinations in Patients with Schizophrenia Spectrum Disorders AbstractBackground and Hypothesis. Auditory-verbal hallucinations (AVH)—the experience of hearing voices in the absence of auditory stimulation—are a cardi

New paper with Tom Whitford, using EEG to investigate inner speech in people with auditory verbal hallucinations in schizophrenia.

academic.oup.com/schizophreni...

It's really not. As described here, N is a subjective self-report. You may as well ask how many fairies people can see dancing on the head of a pin. Conceptually, this is simply not a verifiable performance metric.

SAVE THE DATE! Australasian Society for Experimental Psychology (EPC) & Asia-Pacific Conference on Vision (APCV) Joint Meeting from 1-4 July at the University of Auckland, NZ.
#PsychSciSky #VisionScience #neuroskyence

More information to follow!
visualneuroscience.auckland.ac.nz/epc-apcv-2026/

I so wish : )

Failing that - I'll raise a glass to your continued good health at that time : )

If the DV are RTs - it would be important to control for local image contrasts. If the DV is recognition, controlling for ~all image properties is futile, as these are what we recognize. If you want to know what properties we rely on, well that is a different question (its some of them)

The problem - if there is one, is you didn't control for oriented contrast energy, spatial frequency content, local or long range curvature ect ect... Rotating an image causes big changes in these properties. Deciding if control is futile or sensible depends on context.

Exactly - attention kicks in and re-weights image properties ect - but as you say, the images are cool, and I want a coaster : )

If you are worried that detection or RTs might be related to contrast diffs ect - sure, control for that type of thing. But I think claiming to control for ~all image stats is futile if you still want to be able to recognize things in the image

Obv it depends on context, but if you control for all image stats, you could not recognize - as that depends on image stats we have learnt to associate with meaning. So I never understand when papers claim to have controlled for image stats - as they haven't if people can still recognize things

Bluesky is not a great platform for nuance : )

I also find it really hard to follow conversations here, and think people should use tildes more often

If there is any disagreement - it is with the idea that controlling for low-level confounds is a sensible goal.

The info that is mapped to semantics is correlated image structure, that is changed when the images are reoriented. So it is a super cool demo of anagram images (I want a coaster), but it does not show that 'high-level' effects are driven by identical stimuli. You have to change the stimuli

You just taught me a word : )

Will be looking for opportunities to refer to 'elides" : )

Replications (stroop-like interference, aesthetics shaped by knowledge) tapped understanding entrained by recognition of correlated image structure. This doesn't seem much different to 'house' and 'horse' having different meanings. The task that didn't (a visual search) had a detection component.

Yes, but even with multi-stable images attention acts to re-weight how we process the different features of the image. But at least that is all happening within the brain/mind, and does not rely on people detecting reliable image features

I think attempts to dissociate the cognitive processes entrained by an image from all low-level image properties are misguided, as it is ultimately some minimal set of correlated low-level image properties that allow us to recognize anything (including faces as faces, and rabbits as rabbits)

Only images that are multi-stable without any change (in orientation or anything else) could be said to dissociate meaning from the image structure, but even there attention kicks in, and our brains re-weight the encoded image structure by via selective processing.

The words 'house' and 'horse' are similar, but due to subtle image structure differences it is no surprise they trigger very different associations. Similarly your images are only subtly different when rotated, but they are different and it is not surprising they can trigger different associations

I think you are attempting the impossible. To extract meaning from an image, we detect correlations between image structure and high-level meaning. It is telling that all your tasks that have positive results are high-level, and the only null result comes from a search task that involves detection

By changing the orientation of the image, you have changed the low-level image statistics. Once again you have shown that you cannot manipulate high-level visual properties while holding all low-level content constant

Obviously I have no idea in this space.

When I dream, I am Fully emmersed and embodied in the scene. At least until I guess I am dreaming, which I can conform as I don’t have any sense of touch.

People’s descriptions of imagery don’t sound like that? They sound like they are more selective no?

In my mind, if you have to close your eyes you are not a projector. Whatever representation your brain can generate clearly cannot be projected into the world you are seeing.