Model statistics

Test splitMS (multi-signer) - test set includes signers also seen in training (known to the model).SI (signer-independent) - test set contains only signers unseen in training (generalization test).

Uni-Sign (ensemble)

Transfer source	Split	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	BLEURT	chrF	BERTScore
OpenASL + How2Sign (6 models)	MS	34.62	20.69	14.17	10.36	24.43	44.37	27.89	89.24
OpenASL + How2Sign (2 models)	SI	30.00	15.65	9.85	6.68	18.65	39.19	23.41	88.35

What the ensembles are made of

MS: six Uni-Sign models chosen one by one on the validation set. They are the OpenASL and How2Sign transfer models, each with two random seeds, a model trained on OpenASL, then How2Sign, then PJM, and a model trained on a mix of the references and translations by other models that had not seen those clips.

SI: two Uni-Sign models, with OpenASL and How2Sign transfer.

An ensemble averages the models' predictions at every next word, so it is not a single-model result, and it costs as much computation as it has models.

Uni-Sign

Transfer source	Split	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	BLEURT	chrF	BERTScore
OpenASL	MS	32.61	18.41	12.10	8.53	21.72	42.43	26.04	88.91
How2Sign	MS	32.42	18.33	12.11	8.61	21.90	41.98	25.86	88.86
CSL-News	MS	26.42	12.69	7.70	5.16	16.35	36.36	20.56	87.79
none (from scratch)	MS	18.58	5.77	2.96	1.82	8.48	30.04	13.36	86.14
OpenASL	SI	29.28	14.93	9.18	6.09	17.62	38.16	22.82	88.18
How2Sign	SI	28.78	14.43	8.86	5.91	17.37	37.83	22.36	88.14
CSL-News	SI	21.98	7.89	4.00	2.41	11.36	31.31	16.80	87.01
none (from scratch)	SI	15.29	3.48	1.52	0.85	6.11	30.08	13.07	85.95

SpaMo

Transfer source	Split	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	BLEURT	chrF	BERTScore
PHOENIX	MS	28.87	13.76	8.27	5.54	19.66	35.12	21.06	87.79
none (PJM only)	MS	18.55	6.60	3.40	2.03	12.37	30.25	15.32	86.13
PHOENIX	SI	21.97	7.46	3.63	2.09	12.42	28.84	16.02	86.63
none (PJM only)	SI	17.65	4.53	1.86	0.93	9.28	27.97	13.14	85.75

SpaMo (mod.)

Transfer source	Split	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	BLEURT	chrF	BERTScore
PHOENIX	MS	27.90	13.14	7.76	5.06	18.86	34.30	20.94	87.68
none (PJM only)	MS	18.45	6.04	2.85	1.56	11.92	30.42	14.50	86.02
PHOENIX	SI	21.97	7.39	3.59	2.06	12.26	27.86	16.17	86.53
none (PJM only)	SI	18.78	5.23	2.27	1.17	10.12	27.67	14.14	86.03

Bold marks the best value in each column within its split, MS or SI.