Skip to content
An independent guide to international theatreMany cultures. A shared stage.
The InternationalistsTHEATRE ACROSS BORDERS

A GUIDE TO THEATRE ACROSS BORDERS

Translation & Multilingual Theatre

How Should Theatre Researchers Evaluate Machine Translation for Source Materials and Rehearsal Documents?

THE SHORT ANSWER

Theatre researchers should evaluate machine translation as a task-specific aid, not as an authoritative translator. They should define the document's purpose and risk, apply criteria for accuracy, dramatic function, cultural context and performability, and require qualified human review before rehearsal use. For research materials, machine translation can support triage and access, but quotations and production decisions should rest on checked human translation.

At a glance Quick facts
Core principle
Evaluate machine translation by purpose and consequence of error, not by fluency alone.
Higher-risk use
Rehearsal documents that guide blocking, cues or spoken text require qualified human review.
Key theatrical criteria
Accuracy, dramatic function, cultural context, consistency and performability.
Research methods
Comparative source-target analysis, process observation and production or read-through review.
Documentation
Record tool, version, date, language pair, prompt or settings, reviewer and corrections.
Ethics
Confidential scripts, personal notes and rights correspondence need consent and data-protection checks before cloud processing.

The short answer is that theatre researchers should evaluate machine translation (MT) as a task-specific aid, not as an authoritative translator. Begin with the purpose of the document, set a risk threshold, and test the output against criteria that matter in theatre: accuracy, dramatic function, cultural context, consistency and performability. Source materials used for research may tolerate more risk than rehearsal documents, but neither should be treated as self-validating. Any text that will guide actors, stage management, surtitles or production decisions needs qualified human review.

Definitions: source materials and rehearsal documents

In this context, machine translation means any automated system that converts text from one language into another, including general-purpose translation engines and large language models (LLMs) used through a chat interface or API. Source materials are documents used to understand a play or production context: scripts and extracts, reviews, interviews, programme notes, historical articles, rights correspondence and research dossiers. Rehearsal documents are working papers that shape the production: scene breakdowns, character notes, dramaturgical annotations, blocking notes, cue sheets, surtitle drafts, rehearsal reports and translated extracts distributed to the company.

The distinction is practical rather than absolute. A research dossier can become a rehearsal document as soon as a director circulates it to actors. Researchers should therefore record not only what was translated but also who will use it and what decisions it may influence.

Why theatre is a demanding test case for machine translation

Theatre translation is not a neutral transfer of information. Research on stage-play translation notes that it demands dramatic tone, cultural nuance and metaphorical resonance if the text is to remain performable; a study by Su, Zhao and Li constructed 50 metaphor-rich passages and compared LLM translations with human renderings on criteria including metaphor fidelity and readability (Su, Zhao and Li). Geraldine Brodie describes theatre translation as hyper-specialised because it is focused on a performed text and its users, including actors, directors and audience members, and because it requires both linguistic and performance expertise (Brodie). Marguerite Johnson argues that translation is better understood as cultural infrastructure than as a transparent channel: choices in libretti, surtitles and programme notes shape how performances travel and which audiences can answer them (Johnson).

These points set the evaluation bar. A machine translation may be fluent and still be wrong for the theatre if it loses irony, flattens register, confuses an addressee, breaks a rhythm, misreads a stage direction or embeds a cultural assumption that the production has not chosen. Evaluation should therefore combine textual comparison with attention to use.

A fit-for-purpose evaluation framework

The most defensible approach is a staged, documented assessment. The depth of evaluation should rise with the consequence of error.

  1. State the purpose and consequence of error. Is the translation for private orientation, a funding application, a table-work handout, a surtitle draft or a working script? A misread name in a background article is different from a mistranslated cue in a rehearsal room.
  2. Classify the document and its risk. Separate low-risk orientation material, medium-risk dramaturgical notes and high-risk performance text, legal correspondence or culturally sensitive material. High-risk documents should not be circulated as raw MT.
  3. Build a representative test set. Select passages that contain metaphor, idiom, humour, forms of address, overlapping speech, song or verse, technical theatre terms, historical allusions and ambiguous stage directions. A sample of easy sentences will flatter any system.
  4. Define criteria and a rubric before viewing output. Decide how reviewers will judge accuracy, dramatic function, cultural context, consistency and performability. Use a simple scale and require comments for low scores.
  5. Use mixed methods. Compare source, machine output and human reference; observe how the text is actually used; and review rehearsal or production documents. Glynn notes that theatre-translation research has developed output-oriented and process-oriented methods, including comparative textual analysis, production reviews and ethical reflection (Glynn).
  6. Test with users. A read-through by actors, a stage-management check of cues, or a surtitle timing test can reveal problems that sentence-level review misses.
  7. Record provenance. Note the tool, version, date, language pair, prompt or settings, reviewer, corrections and confidence. LLM output can vary by interface and time, so an unreported translation is hard to evaluate or reproduce.

Criteria that matter in theatre research

DimensionQuestion for reviewersUseful evidence
Accuracy and completenessAre agents, negation, numbers, names and stage directions correct? Have omissions or additions changed the meaning?Segment-level comparison by a bilingual reviewer; annotated corrections
Dramatic functionDoes the line perform the same speech act, tone, subtext and character voice?Translator commentary; read-through; actor feedback
Cultural and historical contextAre allusions, institutions, humour, taboo and register handled appropriately for the intended audience?Glossary; cultural consultant review; source-context notes
PerformabilityIs it speakable, breathable and actable? Does it fit cue timing or surtitle length?Timed read-through; rehearsal observation; surtitle draft test
ConsistencyAre names, pronouns, titles and technical terms stable across documents?Termbase; style sheet; version-controlled files
Ethics and dataIs the material confidential, personal, unpublished or rights-protected? Was consent given for cloud processing?Data-protection review; documented permissions; redaction log

Automatic metrics may be useful for screening large batches, but they are not sufficient evidence of theatrical quality. A high similarity score cannot tell a researcher whether a joke lands, a pronoun is theatrically plausible or a stage direction will mislead a production team.

Evaluating machine translation for source materials

For source materials, MT is often most defensible as an access technology. It can help a researcher decide whether an article is relevant, locate a passage, prepare a glossary or brief a translator. In these uses, the evaluation question is whether the output is good enough for the research task, not whether it is publishable. A researcher might spot-check names, dates and technical terms, keep the source text alongside the MT, and mark uncertain passages for human translation.

When source materials are quoted, published or used to make claims about a play, the standard changes. Quotations should be checked against the source by a qualified bilingual reader or translator. If the material is a play text, the researcher should treat the MT as a draft and commission or produce a human translation of the passages under analysis. The Su, Zhao and Li study is a useful model here because it compares LLM output with human renderings on metaphor-rich theatrical passages rather than relying on general fluency alone (Su, Zhao and Li).

Evaluating machine translation for rehearsal documents

Rehearsal documents carry operational risk. A translated scene breakdown can misidentify who enters when; a translated note can reverse a directorial intention; a rendered stage direction can become a blocking fact. The threshold should therefore be higher and the workflow more social. A useful model is a traffic-light system: green for low-risk lists that a bilingual member of the team can check quickly; amber for dramaturgical notes that need translator review; red for script pages, legal documents, culturally sensitive material and anything that will be spoken, surtitled or used to set cues.

Human review should be role-specific. A qualified translator can judge source-target accuracy and register; a dramaturg can judge context and textual function; a director or actor can judge speakability; a stage manager can judge operational clarity. No single reviewer needs to answer every question, but the final document should show who approved it and what remains uncertain. If a company has no shared language, a bilingual stage manager or interpreter may be the practical bridge, but that person should not be assumed to be a professional translator without agreement.

Common problems in evaluation

  • Treating fluency as fidelity. Smooth output can hide mistranslation, omission or invented detail.
  • Testing isolated sentences. Theatre meaning often depends on adjacency, speaker relationships and staging context.
  • Using back-translation as proof. Back-translation can expose gross errors but cannot confirm theatrical adequacy.
  • Ignoring document purpose. A translation that is adequate for private orientation may be unsafe for rehearsal use.
  • Forgetting provenance. Without tool, date, version and prompt, findings are difficult to reproduce.
  • Skipping user testing. Actors and stage managers reveal problems that desk review misses.
  • Overlooking ethics. Unpublished scripts, personal notes and rights correspondence may not belong in a public translation tool.

Minimum protocol for a small research team

  1. Map the documents, languages, users and decisions at stake.
  2. Redact personal or confidential material and check that processing is permitted.
  3. Select representative passages for a reference translation by a qualified human translator.
  4. Run the machine translation, label it clearly as MT and retain the source.
  5. Review blind where possible, using a shared rubric and written comments.
  6. Test key outputs in a read-through, rehearsal or production-document check.
  7. Record corrections, unresolved uncertainties and the basis for approval.
  8. Report limitations, including sample size, language pair, tool version and reviewer expertise.

How to report machine-translation evaluation

Researchers should report enough detail for readers to judge the claim. State the languages, document types, tool and version, date of use, interface or prompt, sample selection and reviewer qualifications. Distinguish between accuracy, readability and performability rather than collapsing them into one score. Where human and machine outputs are compared, explain how the human reference was produced and whether reviewers were blind. Do not present a single percentage as if it settled theatrical quality. Report examples of disagreement and explain how the translation was used in practice.

The broader point is that evaluation is a dramaturgical and ethical act, not only a technical one. Machine translation can widen access to source materials and help multilingual teams prepare. It becomes risky when its fluency is mistaken for authority. The defensible question is not whether a machine can translate theatre, but whether a particular translation is fit for a particular purpose, reviewed by the right people and documented honestly enough for others to assess.

FAQ

Can machine translation be used in rehearsal?

Yes, as a clearly labelled aid, but not as an unreviewed final text. Low-risk lists may need only a bilingual check; script pages, cues, surtitles and culturally sensitive notes need qualified human review and, where possible, a read-through.

Are automatic metrics enough to evaluate theatre translation?

No. They can help screen large batches but do not reliably measure dramatic function, cultural nuance, speakability or rehearsal usability.

How should researchers evaluate metaphor in machine-translated theatre texts?

Compare the machine output with a human rendering and with the source passage, asking whether the metaphor still performs its dramatic function. The Su, Zhao and Li study assessed metaphor fidelity and readability in 50 metaphor-rich passages, which is a useful design to adapt.

What should be recorded when using machine translation?

Tool, version, date, languages, prompt or settings, document type, reviewer, corrections and any unresolved uncertainty. This makes findings auditable and reproducible.

Should researchers compare machine translation with human translation?

Yes, but the human reference must be fit for purpose and reviewers should assess separate criteria rather than relying on a single score.

READ FURTHER

Sources & references

  1. Su, Chiyuan; Zhao, Eric; and Li, Zhixiang. Evaluating Large Language Models Against Human in Stage Play Translation. OSF Preprints, 23 September 2025. https://doi.org/10.31235/osf.io/j6xb4_v1
  2. Johnson, Marguerite. Translation as Cultural Infrastructure in an Age of Machine Mediation. Opera News: Journal of Music and Performing Arts, 28 February 2025. https://www.operanews.org/articles/2025/volume-2025-issue-2/translation-as-cultural-infrastructure-in-an-age-of-machine-mediation-4bad75
  3. Glynn, Dominic. Theater Translation Research Methodologies. International Journal of Qualitative Methods, 2020. https://doi.org/10.1177/1609406920937146
  4. Brodie, Geraldine. Translating for the Theatre. UCL Discovery. https://discovery.ucl.ac.uk/id/eprint/10145363/3/Brodie_TranslatingfortheTheatre_chapter_AAM.pdf

THE LANGUAGE OF PRACTICE

Related terms

Machine translation
Automated conversion of text between languages by software or models.
Theatre translation
Translation for performance, including scripts, adaptation, surtitles and rehearsal materials.
Performability
The extent to which a translated text can be spoken, acted and staged effectively.
Back-translation
Retranslating a target text into the source language to check meaning; useful for spotting errors but not proof of theatrical adequacy.
Rehearsal documentation
Working records such as scene breakdowns, notes, cue sheets and reports used in preparation.
Source text
The original text from which a translation is made.

A SHARED BODY OF KNOWLEDGE

Something to add or correct? Help us keep the conversation informed.

Corrections & updates

Leave a Reply

Your email address will not be published. Required fields are marked *

THE CONVERSATION CONTINUES

Keep a window open to the world.

Fresh perspectives and practical reading, straight to your feed reader.

Subscribe via RSS

FOLLOW YOUR CURIOSITY

Find your next idea.

Search the journal, practical guides, and pages.

THE PRACTICAL TOOLKIT / 01

Your first conversations.

Use these prompts to begin planning with your collaborators. Your progress is saved in this browser.

A starting point for discussion. Adapt it to your project and check local requirements with the relevant organisations.

THE PRACTICAL TOOLKIT / 02

A time to come together.

Choose a date and time in UTC to see when your collaborators would meet. Times account for daylight saving on your chosen date.

Shaded times fall outside 09:00–18:00 local time. Agree working hours and access needs with your team.

BEHIND THE IMAGES

Photography credits

Editorial stock photographs illustrate themes; they do not document the productions discussed on this site. Used under the Pexels licence.