Ben Jonathan Wagner

Interacting cortico-basal ganglia-thalamocortical loops shape behavioral control through cognitive maps and shortcuts Control of behavior is often explained in terms of a dichotomy, with distinct neural circuits underlying goal-directed and habitual control, yet accumulating evidence suggests these processes are deep...

I think this paper is a good read from a neuroscience perspective on how repetition and goal-directed reward learning are intertwined, resulting in short-cuts that correspond to decision biases as a function of task complexity. www.cell.com/trends/neuro...

I would agree that this is an argument. However, a "flat curve" could also be the equilibrium between repetition and exploration (because deciding between arbitrary symbols without feedback is quite boring and participants start to explore). But sure, one needs to investigate...

In those tasks the Range model is not qualitatively different from REL. But sure, one can look at this. In general the ABS model is not bad in some datasets but repetitions become more important as task complexity increases. I wouldn´t say that the signature in (c) is captured well by ABS though.

I am not sure if I understand this point correctly, but we also tested the model on six datasets without transfer feedback.

I would argue that if the preference mechanism is rather associative, likely due to some form of policy compression and/or WM. In free choice, this is related to agency; however, in observational learning, this can happen without agency (e.g., via strengthening an associative context-stimulus link).

Dear Stefano, I appreciate the discussion. We actually present true ex-novo simulations for some tasks in the supplement of the paper. Those simulations show that when we directly compare normalization vs. repetition (via task design), the results match our empirical findings very well.

Finally..., thank you :)

Out now in Translational Psychiatry! www.nature.com/articles/s41...

Many thanks also to @stepalminteri.bsky.social, @sophiebavard.bsky.social, and @gjocham.bsky.social for being helpful and for promptly answering all the questions I had.

As a side note, I would not interpret our results as showing that relative value learning (or specific forms thereof) does not exist, but rather that it may not be the primary force behind preference biases in such tasks.

We therefore believe that, in the end, repetition may be a more important factor in shaping choice than previously acknowledged.

Of course, the idea of repetition biases is not new in RL, but to our knowledge it has not yet been shown that such a mechanism can sufficiently and consistently account for such preference biases across a range of value-learning tasks.

Conceptually, I think this aligns very well with work on policy compression by @lucylai.bsky.social and @gershbrain.bsky.social and recent work by @annecollins.bsky.social.

Notably even when Q-value differences or objective absolute or relative value differences were absent. Overall, the impact of this repetition mechanism is larger in more complex tasks.

This holds both in standard analyses and in hierarchical Bayesian modeling, and importantly in settings where the two accounts make divergent predictions. We also found that post-task valuation ratings show that participants rated stimuli higher when they had been chosen more often.

Action repetition biases choice in context-dependent decision-making - Communications Psychology This study shows that decision biases previously attributed to value normalization (e.g. relative value learning or range normalization) are better explained by action repetition. Repeating an action ...

Very happy that this is out www.nature.com/articles/s44.... Together with @stefankiebel.bsky.social we show that decision biases in context-dependent decision making, previously attributed to different forms of value normalization, are very well explained by habit-like action repetition.

A habit and working memory model as an alternative account of human reward-based learning Nature Human Behaviour - In this study, Collins proposes an alternative dual-process (working memory and habit) model of reinforcement learning in humans.

My paper is out!
Computational modeling of error patterns during reward-based learning show evidence that habit learning (value free!) supplements working memory in 7 human data sets.
rdcu.be/eQjLN

A new preprint 📝 with @tobiasuhauser.bsky.social @kenzakdr.bsky.social @benjwagner.bsky.social
and Andrew Webb accompanying our cpm-toolbox.net python modelling library - including details about our motivations, toolbox features, framework and workflows!

👉 osf.io/preprints/ps...

Release v0.23.18 - New Prospect Models, improved parameter management, and a few bug fixes · DevComPsy/cpm Install You can install the new release straight from the PyPi repository: pip install cpm-toolbox Added Add input validation and error handling in all cpm.optimisation.minimise methods Add test u...

New models (three variants of Prospect Theory), new features (more ways to manage parameters, more model components to use), and of course bug fixes. If you want to make your computational modelling reproducible and robust, check out and install the new version of *cpm*:

github.com/DevComPsy/cp...

I'm wondering, do you use chatgpt or other ai tools at all? Or do you use them in a "critical way"?

@stefankiebel.bsky.social

Cognitive computational model reveals repetition bias in a sequential decision-making task - Communications Psychology Using a sequential decision making task and cognitive modeling, we show that human decisions are best explained by a combination of repetition bias and goal directed reward-based behavior.

Using a sequential decision making task and cognitive modelling, this study shows that human decisions are best explained by a combination of repetition bias and goal directed reward-based behavior.
@benjwagner.bsky.social
www.nature.com/articles/s44...

Cognitive computational model reveals repetition bias in a sequential decision-making task - Communications Psychology Using a sequential decision making task and cognitive modeling, we show that human decisions are best explained by a combination of repetition bias and goal directed reward-based behavior.

www.nature.com/articles/s44...

can you post everything over here? thank you!

Just deactivated my X account.

Participants enhanced or inhibited their habitual responses based on whether they were congruent or incongruent with goal-directed behavior.
Using drift-diffusion modeling, we found that habitual and goal-directed response tendencies interact on the level of evidence accumulation (drift-rate).

We discovered that the influence of a habit isn’t static, it depends on the number of repetitions of an action sequence.
🧠 Approximately 60% of participants adaptively adjusted their habitual responses according to the task context.

Context-Dependent Interaction Between Goal-Directed and Habitual Control Under Time Pressure Habits are an important aspect of human behaviour. Habits are reflexive, inflexible, and fast, in contrast to goal-directed behaviour which is reflective, flexible, and slow. Current theories assume t...

In our new preprint @saschafrolich.bsky.social , @MichaelSmolka & @StefanKiebel on how "habits interact with goal-directed behavior under time pressure", we found that habitual behavior varies as a function of context and repetition 🔄 www.biorxiv.org/content/10.1...

Chronic Deep Brain Stimulation of the Human Nucleus Accumbens Region Disrupts the Stability of Inter... When choosing between rewards that differ in temporal proximity (intertemporal choice), human preferences are typically stable, constituting a clinically relevant transdiagnostic trait. Here we show, ...

Oh, thank you. It seems it is not working. This one should: www.jneurosci.org/content/43/4...

4/4 However, further research is needed to clarify a causal link.⚡