Early cancer diagnosis via interpretable two-layer machine learning of plasma extracellular vesicle long RNA

Table 1 Dataset information

Sample	Dataset	Type	Number of samples
			Male	Female	Both
Non-normal	S1non-normal	Pancreatic ductal adenocarcinoma	167	117	284
	S2non-normal	Chronic pancreatitis	73	27	100
Normal	Snormal	Healthy	71	46	117

Table 2 Performance of the first-layer models on training dataset

Models	ACC, %	Sen, %	Sp, %	MCC	AUC
PDACandCPvsHealth-MDA-RF (1-1)	88.68	96.30	64.00	0.6674	0.9328
PDACandCPvsHealth-MDA-SVM (1-2)	90.57	93.83	80.00	0.7383	0.9146
PDACandCPvsHealth-MDA-DL (1-3)	87.74	88.89	84.00	0.6869	0.9151
PDACandCPvsHealth-MDA-XGB (1-4)	87.74	95.06	64.00	0.6408	0.9304
PDACandCPvsHealth-SVD PCA Index-RF (1-5)	96.23	100.00	84.00	0.8947	0.9901
PDACandCPvsHealth-SVD PCA Index-SVM (1-6)	85.85	95.06	56.00	0.5773	0.9200
PDACandCPvsHealth-SVD PCA Index-DL (1-7)	89.62	90.12	88.00	0.7363	0.9580
PDACandCPvsHealth-SVD PCA Index-XGB (1-8)	91.51	98.77	68.00	0.7549	0.9294
PDACandCPvsHealth-FastHCS Index-RF (1-9)	89.62	96.30	68.00	0.6976	0.9462
PDACandCPvsHealth-FastHCS Index-SVM (1-10)	88.68	96.30	64.00	0.6674	0.9230
PDACandCPvsHealth-FastHCS Index-DL (1-11)	82.08	88.89	60.00	0.4959	0.8993
PDACandCPvsHealth-FastHCS Index-XGB (1-12)	85.85	93.83	60.00	0.5840	0.9407
PDACandCPvsHealth-Nipals PCA Index-RF (1-13)	89.62	98.77	60.00	0.6969	0.9059
PDACandCPvsHealth-Nipals PCA Index-SVM (1-14)	89.62	97.53	64.00	0.6957	0.9294
PDACandCPvsHealth-Nipals PCA Index-DL (1-15)	90.57	95.06	76.00	0.7319	0.9679
PDACandCPvsHealth-Nipals PCA Index-XGB (1-16)	91.51	98.77	68.00	0.7549	0.9294
PDACandCPvsHealth-Probabilistic PCA Index-RF (1-17)	93.40	96.30	84.00	0.8145	0.9763
PDACandCPvsHealth-Probabilistic PCA Index-SVM (1-18)	91.51	92.59	88.00	0.7760	0.9560
PDACandCPvsHealth-Probabilistic PCA Index-DL (1-19)	88.68	90.12	84.00	0.7059	0.9462
PDACandCPvsHealth-Probabilistic PCA Index-XGB (1-20)	86.79	93.83	64.00	0.6159	0.9007

ACC: Accuracy; Sen: Sensitivity; Sp: Specificity; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.

Table 3 Performance of the first-layer models on independent dataset

Models	ACC, %	Sen, %	Sp, %	MCC	AUC
PDACandCPvsHealth-MDA-RF (1-1)	91.33	98.26	68.57	0.7467	0.9606
PDACandCPvsHealth-MDA-SVM (1-2)	87.33	94.78	62.86	0.6257	0.9409
PDACandCPvsHealth-MDA-DL (1-3)	84.00	87.83	71.43	0.5714	0.9153
PDACandCPvsHealth-MDA-XGB (1-4)	90.67	96.52	71.43	0.7278	0.9595
PDACandCPvsHealth-SVD PCA Index-RF (1-5)	90.67	97.39	68.57	0.7262	0.9288
PDACandCPvsHealth-SVD PCA Index-SVM (1-6)	88.67	96.52	62.86	0.6635	0.9198
PDACandCPvsHealth-SVD PCA Index-DL (1-7)	85.33	86.09	82.86	0.6363	0.9190
PDACandCPvsHealth-SVD PCA Index-XGB (1-8)	90.67	98.26	65.71	0.7261	0.9160
PDACandCPvsHealth-FastHCS Index-RF (1-9)	84.00	93.91	51.43	0.5146	0.8737
PDACandCPvsHealth-FastHCS Index-SVM (1-10)	87.33	94.78	62.86	0.6257	0.9155
PDACandCPvsHealth-FastHCS Index-DL (1-11)	84.67	90.43	65.71	0.5672	0.9016
PDACandCPvsHealth-FastHCS Index-XGB (1-12)	86.67	93.91	62.86	0.6080	0.9058
PDACandCPvsHealth-Nipals PCA Index-RF (1-13)	90.67	98.26	65.71	0.7261	0.8961
PDACandCPvsHealth-Nipals PCA Index-SVM (1-14)	90.67	99.13	62.86	0.7276	0.9324
PDACandCPvsHealth-Nipals PCA Index-DL (1-15)	88.00	91.30	77.14	0.6716	0.9093
PDACandCPvsHealth-Nipals PCA Index-XGB (1-16)	90.67	98.26	65.71	0.7261	0.9160
PDACandCPvsHealth-Probabilistic PCA Index-RF (1-17)	91.33	96.52	74.29	0.7487	0.9165
PDACandCPvsHealth-Probabilistic PCA Index-SVM (1-18)	93.33	95.65	85.71	0.8137	0.9717
PDACandCPvsHealth-Probabilistic PCA Index-DL (1-19)	89.33	92.17	80.00	0.7081	0.9347
PDACandCPvsHealth-Probabilistic PCA Index-XGB (1-20)	89.33	93.04	77.14	0.7019	0.9426

ACC: Accuracy; Sen: Sensitivity; Sp: Specificity; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.

Table 4 The results of the first-layer models five-fold cross-validation

Models	ACC, %	MCC	AUC
PDACandCPvsHealth-MDA-RF (1-1)	88.61	0.6710	0.9427
PDACandCPvsHealth-MDA-SVM (1-2)	85.74	0.5850	0.9059
PDACandCPvsHealth-MDA-DL (1-3)	83.19	0.5267	0.8910
PDACandCPvsHealth-MDA-XGB (1-4)	89.17	0.6858	0.9475
PDACandCPvsHealth-SVD PCA Index-RF (1-5)	88.90	0.6724	0.9288
PDACandCPvsHealth-SVD PCA Index-SVM (1-6)	92.31	0.7802	0.9760
PDACandCPvsHealth-SVD PCA Index-DL (1-7)	89.72	0.7130	0.9471
PDACandCPvsHealth-SVD PCA Index-XGB (1-8)	88.89	0.6753	0.9472
PDACandCPvsHealth-FastHCS Index-RF (1-9)	84.60	0.5352	0.9025
PDACandCPvsHealth-FastHCS Index-SVM (1-10)	85.73	0.5670	0.9268
PDACandCPvsHealth-FastHCS Index-DL (1-11)	83.73	0.5386	0.8951
PDACandCPvsHealth-FastHCS Index-XGB (1-12)	86.90	0.6230	0.9250
PDACandCPvsHealth-Nipals PCA Index-RF (1-13)	88.02	0.6540	0.9274
PDACandCPvsHealth-Nipals PCA Index-SVM (1-14)	87.46	0.6306	0.9334
PDACandCPvsHealth-Nipals PCA Index-DL (1-15)	88.86	0.6957	0.9336
PDACandCPvsHealth-Nipals PCA Index-XGB (1-16)	88.89	0.6753	0.9474
PDACandCPvsHealth-Probabilistic PCA Index-RF (1-17)	89.16	0.6796	0.9478
PDACandCPvsHealth-Probabilistic PCA Index-SVM (1-18)	91.19	0.7430	0.9389
PDACandCPvsHealth-Probabilistic PCA Index-DL (1-19)	89.47	0.7163	0.9576
PDACandCPvsHealth-Probabilistic PCA Index-XGB (1-20)	88.61	0.6686	0.9106

ACC: Accuracy; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.

Table 5 Performance of the second-layer models on training dataset

Models	ACC, %	Sen, %	Sp, %	MCC	AUC
PDACvsCP-MDA-RF (2-1)	90.12	95.00	76.19	0.7363	0.9444
PDACvsCP-MDA-SVM (2-2)	93.83	96.67	85.71	0.8372	0.9563
PDACvsCP-MDA-DL (2-3)	85.19	90.00	71.43	0.6143	0.9159
PDACvsCP-MDA-XGB (2-4)	86.42	91.67	71.43	0.6412	0.9246
PDACvsCP-SVD PCA Index-RF (2-5)	93.83	96.67	85.71	0.8372	0.9802
PDACvsCP-SVD PCA Index-SVM (2-6)	93.83	98.33	80.95	0.8357	0.9627
PDACvsCP-SVD PCA Index-DL (2-7)	88.89	90.00	85.71	0.7266	0.9159
PDACvsCP-SVD PCA Index-XGB (2-8)	86.42	90.00	76.19	0.6521	0.9603
PDACvsCP-FastHCS Index-RF (2-9)	93.83	98.33	80.95	0.8357	0.9563
PDACvsCP-FastHCS Index-SVM (2-10)	88.89	93.33	76.19	0.7065	0.9750
PDACvsCP-FastHCS Index-DL (2-11)	93.83	96.67	85.71	0.8372	0.9333
PDACvsCP-FastHCS Index-XGB (2-12)	93.83	98.33	80.95	0.8357	0.9357
PDACvsCP-Nipals PCA Index-RF (2-13)	92.59	93.33	90.48	0.8145	0.9651
PDACvsCP-Nipals PCA Index-SVM (2-14)	87.65	93.33	71.43	0.6698	0.9397
PDACvsCP-Nipals PCA Index-DL (2-15)	88.89	91.67	80.95	0.7155	0.9246
PDACvsCP-Nipals PCA Index-XGB (2-16)	86.42	90.00	76.19	0.6521	0.9603
PDACvsCP-Probabilistic PCA Index-RF (2-17)	93.83	95.00	90.48	0.8422	0.9698
PDACvsCP-Probabilistic PCA Index-SVM (2-18)	96.30	100.00	85.71	0.9035	0.9833
PDACvsCP-Probabilistic PCA Index-DL (2-19)	91.36	96.67	76.19	0.7680	0.9579
PDACvsCP-Probabilistic PCA Index-XGB (2-20)	90.12	91.67	85.71	0.7520	0.9563

ACC: Accuracy; Sen: Sensitivity; Sp: Specificity; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.

Table 6 Performance of the second-layer models on independent dataset

Models	ACC, %	Sen, %	Sp, %	MCC	AUC
PDACvsCP-MDA-RF (2-1)	88.70	97.65	63.33	0.6931	0.9500
PDACvsCP-MDA-SVM (2-2)	91.30	97.65	73.33	0.7670	0.9549
PDACvsCP-MDA-DL (2-3)	89.57	90.59	86.67	0.7434	0.9282
PDACvsCP-MDA-XGB (2-4)	86.09	92.94	66.67	0.6257	0.9149
PDACvsCP-SVD PCA Index-RF (2-5)	93.04	95.29	86.67	0.8196	0.9641
PDACvsCP-SVD PCA Index-SVM (2-6)	93.04	92.94	93.33	0.8302	0.9753
PDACvsCP-SVD PCA Index-DL (2-7)	86.09	84.71	90.00	0.6888	0.9392
PDACvsCP-SVD PCA Index-XGB (2-8)	92.17	95.29	83.33	0.7951	0.9702
PDACvsCP-FastHCS Index-RF (2-9)	86.96	92.94	70.00	0.6521	0.9380
PDACvsCP-FastHCS Index-SVM (2-10)	90.43	90.59	90.00	0.7691	0.9561
PDACvsCP-FastHCS Index-DL (2-11)	86.96	89.41	80.00	0.6738	0.9149
PDACvsCP-FastHCS Index-XGB (2-12)	85.22	91.76	66.67	0.6053	0.9145
PDACvsCP-Nipals PCA Index-RF (2-13)	93.91	96.47	86.67	0.8407	0.9714
PDACvsCP-Nipals PCA Index-SVM (2-14)	86.96	94.12	66.67	0.6471	0.9604
PDACvsCP-Nipals PCA Index-DL (2-15)	86.96	89.41	80.00	0.6738	0.9192
PDACvsCP-Nipals PCA Index-XGB (2-16)	92.17	95.29	83.33	0.7951	0.9702
PDACvsCP-Probabilistic PCA Index-RF (2-17)	94.78	97.65	86.67	0.8626	0.9741
PDACvsCP-Probabilistic PCA Index-SVM (2-18)	94.78	100.00	80.00	0.8644	0.9255
PDACvsCP-Probabilistic PCA Index-DL (2-19)	86.96	91.76	73.33	0.6582	0.8859
PDACvsCP-Probabilistic PCA Index-XGB (2-20)	92.17	95.29	83.33	0.7951	0.9722

ACC: Accuracy; Sen: Sensitivity; Sp: Specificity; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.

Table 7 The results of the second-layer models five-fold cross-validation

Models	ACC, %	MCC	AUC
PDACvsCP-MDA-RF (2-1)	89.60	0.7239	0.9431
PDACvsCP-MDA-SVM (2-2)	91.44	0.7739	0.9573
PDACvsCP-MDA-DL (2-3)	89.21	0.7293	0.9493
PDACvsCP-MDA-XGB (2-4)	88.09	0.6791	0.9157
PDACvsCP-SVD PCA Index-RF (2-5)	89.60	0.7209	0.9066
PDACvsCP-SVD PCA Index-SVM (2-6)	91.42	0.7796	0.9298
PDACvsCP-SVD PCA Index-DL (2-7)	85.14	0.6272	0.8874
PDACvsCP-SVD PCA Index-XGB (2-8)	82.54	0.5395	0.8957
PDACvsCP-FastHCS Index-RF (2-9)	87.38	0.6612	0.9071
PDACvsCP-FastHCS Index-SVM (2-10)	88.11	0.6882	0.9345
PDACvsCP-FastHCS Index-DL (2-11)	87.37	0.6672	0.9063
PDACvsCP-FastHCS Index-XGB (2-12)	86.62	0.6449	0.9117
PDACvsCP-Nipals PCA Index-RF (2-13)	89.22	0.7158	0.9130
PDACvsCP-Nipals PCA Index-SVM (2-14)	91.06	0.7684	0.9109
PDACvsCP-Nipals PCA Index-DL (2-15)	86.25	0.6401	0.8944
PDACvsCP-Nipals PCA Index-XGB (2-16)	82.54	0.5395	0.8964
PDACvsCP-Probabilistic PCA Index-RF (2-17)	98.13	0.9520	0.9956
PDACvsCP-Probabilistic PCA Index-SVM (2-18)	92.94	0.8195	0.9502
PDACvsCP-Probabilistic PCA Index-DL (2-19)	88.11	0.6951	0.9519
PDACvsCP-Probabilistic PCA Index-XGB (2-20)	82.91	0.5606	0.9011

ACC: Accuracy; MCC: Mathew’s correlation coefficient; AUC: Area under the receiver operating characteristic curve; PDAC: Pancreatic ductal adenocarcinoma; CP: Chronic pancreatitis; MDA: Mean decrease in accuracy; RF: Random forest; SVM: Support vector machine; DL: Deep learning; XGB: Extreme gradient boosting; SVD: Singular value decomposition; PCA: Principal component analysis; FastHCS: High-dimensional congruent subsets; Nipals: Nonlinear iterative partial least squares.