# The Language-Based Assessment Model (L-BAM) Library

The L-BAM library comprises information about pre-trained models, including information about where they are located online so that they can be automatically downloaded in R. The models are continuously being tested using GitHub actions, and you can checkout the code for running each model here. The models can be called with `textPredict()`, `textAssess()` or `textClassify()` like this:

    library(text)

    # Example calling a model using the URL
    textPredict(
      model_info = "valence_facebook_mxbai23_eijsbroek2024",
      texts = "what is the valence of this text?"
    )

    # Example calling a model having an abbreviation
    textClassify(
      model_info = "implicitpower_roberta23_nilsson2024",
      texts = "It looks like they have problems collaborating."
    )

The text prediction functions can be given a model and a text, and automatically transform the text to word embeddings and produce estimated scores or probabilities.

Important: Language-based assessments can be developed in one context—such as social media—and applied in another—like clinical interviews. However, models don’t always generalize across settings. A model’s performance depends on several factors, including the context in which it was trained, the population, the distribution of the psychological outcome, and the language domain (i.e., how similar the language in the training data is to the new data).

Because of this, users are responsible for evaluating whether a model is appropriate for their specific use case. This means checking whether the training and evaluation conditions align with their own data—and, if needed, validating the model’s performance on a subset of their own data before making any conclusions. That’s why each model in the L-BAM Library comes with detailed documentation on its training data, performance metrics, and development process. Transparent documentation helps users make informed decisions and supports reproducible, trustworthy research (for more information see Nilsson et al., in progress).

If you want to add a pre-trained model to the L-BAM library, please fill out the details in this Google sheet and email us (rtext.contact@gmail.com) so that we can update the table online.

Note that you can adjust the width of the columns when scrolling the table.

> Responsible Sharing and Use of Language-Based Assessment Models

> Guidelines for contributors
> To support transparency, traceability, and responsible use, contributors should:
> - Fill out the L-BAM Submission Sheet: Complete the submission form at r-text.org/articles/LBAM.html.
> Include key metadata such as model name, outcome variable(s), model type, training data size, validation metrics, ethical considerations, and relevant meta data (e.g., links to papers or preprints).
> - Public Hosting: Host the model on a publicly accessible platform with version control (e.g., OSF, Hugging Face, GitHub, Bitbucket) to ensure reproducibility and long-term access.
> - Contact: Email a library maintainer (see contact details at r-text.org/articles/LBAM.html) using the same email address provided under `contact_details` in the metadata.

> Once these steps are completed, the model will be published in the L-BAM library, making it accessible to the broader research community.

> Guidelines for users
> Before using a model from the L-BAM library, we recommend the following steps to ensure its suitability for your research context:
> - Verify Source and Contact Information: Review the listed contact details. If in doubt, reach out to the model contributor for clarification.
> - Using models may carry security risks (e.g., malicious code).
> - Each model must include a designated contact person with a valid email address (not placeholder/fabricated) and must be hosted on a platform with version control (e.g., OSF) for transparency and traceability.
> - Always review and trust the source before loading a model. For extra caution, consider loading models in secure, isolated environments (e.g., Docker).
> - Check for linked preprints or peer-reviewed publications describing the model.
> - Critically Evaluate Development and Validation Details: Examine how the model has been validated, including reported performance metrics (e.g., RMSE, correlations) and the populations used. Ensure these align with your use case.
> - Test Generalisability: Apply the model to new data in your context (ideally with criterion variables available for at least a subset, e.g., 100 out of 10,000 texts). This allows you to directly test generalisability.

### Overview of L-BAM pipelines

Training and using models from the L-BAM library involves three key steps. First, written language is turned into numerical formats known as word embeddings using a large language model (Figure 1A). Next, these embeddings are used to build a predictive model linked to a specific outcome or assessment target (Figure 1B). Finally, the trained model can be applied to new texts for evaluation or classification purposes (Figure 1C).

You can find a detailed guide on how to transform language into embeddings and train L-BAMs using the text package in Kjell et al. (2023). Below, we briefly introduce the embedding and training process before showing how to apply models from the L-BAM library.

Figure from Nilsson et al. (in progress).

### References

Gu, Kjell, Schwartz & Kjell. (2024). Natural Language Response Formats for Assessing Depression and Worry with Large Language Models: A Sequential Evaluation with Model Pre-registration.

Kjell, O. N., Sikström, S., Kjell, K., & Schwartz, H. A. (2022). Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy. Scientific reports, 12(1), 3918.

Nilsson, Runge, Ganesan, Lövenstierne, Soni & Kjell (2024) Automatic Implicit Motives Codings are at Least as Accurate as Humans’ and 99% Faster