Classification Report for Nemotron 70B

Invalid Model Output Count

Test	Invalid Count
base	11
0%_64	2
0%_128	1
0%_256	4
0%_512	7
0%_8192	2
0%_32768	19
25%_64	4
25%_128	7
25%_256	6
25%_512	3
25%_1024	2
25%_2048	2
25%_4096	3
25%_16384	1
25%_32768	9
50%_64	7
50%_128	6
50%_256	3
50%_512	2
50%_1024	2
50%_2048	1
50%_4096	2
50%_16384	4
50%_32768	1
75%_64	5
75%_128	6
75%_256	6
75%_512	5
75%_1024	15
75%_2048	3
75%_8192	3
75%_16384	2
75%_32768	2
100%_64	4
100%_128	2
100%_256	5
100%_512	7
100%_1024	14
100%_2048	2
100%_4096	1
100%_8192	1
100%_16384	2
100%_32768	1

Consolidated Classification Report

Test	Total Samples	True Violations	Predicted Violations	Accuracy	Precision (macro)	Recall (macro)	F1-score (macro)	Precision (weighted)	Recall (weighted)	F1-score (weighted)
base	400	239	214	0.852	0.847	0.859	0.85	0.862	0.852	0.854
0%_64	409	246	192	0.77	0.777	0.788	0.769	0.801	0.77	0.772
0%_128	410	247	195	0.761	0.767	0.778	0.76	0.79	0.761	0.763
0%_256	407	244	173	0.742	0.763	0.768	0.742	0.79	0.742	0.743
0%_512	404	242	140	0.663	0.714	0.702	0.662	0.747	0.663	0.659
0%_1024	411	248	97	0.569	0.672	0.629	0.558	0.712	0.569	0.543
0%_2048	411	248	13	0.418	0.625	0.516	0.329	0.671	0.418	0.278
0%_4096	411	248	6	0.406	0.617	0.507	0.305	0.661	0.406	0.25
0%_8192	409	247	12	0.416	0.618	0.514	0.325	0.663	0.416	0.273
0%_16384	411	248	10	0.416	0.652	0.515	0.322	0.703	0.416	0.27
0%_32768	392	231	5	0.423	0.708	0.511	0.315	0.76	0.423	0.266
25%_64	407	245	227	0.848	0.84	0.85	0.844	0.853	0.848	0.849
25%_128	404	244	226	0.847	0.839	0.849	0.842	0.852	0.847	0.848
25%_256	405	243	226	0.844	0.837	0.847	0.84	0.85	0.844	0.846
25%_512	408	245	230	0.86	0.853	0.862	0.856	0.864	0.86	0.861
25%_1024	409	246	237	0.822	0.813	0.819	0.815	0.824	0.822	0.822
25%_2048	409	248	196	0.746	0.751	0.762	0.744	0.775	0.746	0.749
25%_4096	408	247	189	0.73	0.739	0.749	0.729	0.765	0.73	0.733
25%_8192	411	248	227	0.788	0.781	0.79	0.783	0.796	0.788	0.79
25%_16384	410	248	240	0.751	0.74	0.744	0.742	0.754	0.751	0.752
25%_32768	402	241	146	0.639	0.68	0.673	0.639	0.71	0.639	0.636
50%_64	404	241	220	0.834	0.828	0.838	0.831	0.841	0.834	0.835
50%_128	405	245	230	0.854	0.846	0.856	0.85	0.859	0.854	0.855
50%_256	408	246	234	0.863	0.855	0.863	0.858	0.866	0.863	0.863
50%_512	409	246	228	0.848	0.841	0.851	0.844	0.854	0.848	0.849
50%_1024	409	246	238	0.848	0.841	0.846	0.843	0.85	0.848	0.849
50%_2048	410	247	226	0.768	0.761	0.769	0.763	0.776	0.768	0.77
50%_4096	409	246	228	0.77	0.762	0.77	0.764	0.776	0.77	0.772
50%_8192	411	248	190	0.727	0.737	0.746	0.726	0.762	0.727	0.73
50%_16384	407	247	173	0.676	0.696	0.701	0.675	0.725	0.676	0.678
50%_32768	410	247	100	0.598	0.703	0.657	0.588	0.746	0.598	0.575
75%_64	406	244	232	0.857	0.85	0.857	0.853	0.86	0.857	0.858
75%_128	405	244	229	0.844	0.837	0.846	0.84	0.849	0.844	0.845
75%_256	405	243	234	0.854	0.847	0.853	0.85	0.856	0.854	0.855
75%_512	406	245	229	0.847	0.84	0.849	0.843	0.852	0.847	0.848
75%_1024	396	241	226	0.806	0.796	0.805	0.799	0.811	0.806	0.807
75%_2048	408	246	221	0.787	0.78	0.79	0.782	0.797	0.787	0.789
75%_4096	411	248	226	0.747	0.739	0.747	0.741	0.755	0.747	0.749
75%_8192	408	245	144	0.684	0.734	0.722	0.683	0.768	0.684	0.68
75%_16384	409	246	109	0.592	0.678	0.645	0.584	0.715	0.592	0.573
75%_32768	409	246	110	0.589	0.673	0.642	0.582	0.71	0.589	0.571
100%_64	407	245	227	0.843	0.835	0.845	0.839	0.848	0.843	0.844
100%_128	409	246	231	0.861	0.853	0.862	0.857	0.865	0.861	0.862
100%_256	406	245	230	0.865	0.857	0.866	0.861	0.869	0.865	0.865
100%_512	404	244	233	0.844	0.836	0.843	0.839	0.847	0.844	0.845
100%_1024	397	244	227	0.806	0.796	0.806	0.799	0.812	0.806	0.808
100%_2048	409	247	227	0.804	0.797	0.806	0.799	0.811	0.804	0.806
100%_4096	410	247	253	0.746	0.735	0.732	0.734	0.745	0.746	0.745
100%_8192	410	247	198	0.744	0.748	0.758	0.742	0.77	0.744	0.747
100%_16384	409	247	161	0.682	0.714	0.713	0.682	0.745	0.682	0.682
100%_32768	410	247	260	0.705	0.691	0.685	0.687	0.702	0.705	0.703