Objective:To compare the automatic modeling effects of Weka, Rapidminer, and Ymodel

Data to be used:5 pieces of data in total, 3 pieces of classification, and 2 pieces of regression

2 classic Kaggle cases and 3 real business data

Due to the limited data size of Rapidminer's free version of 10000 items, three real business data was sampled, with sample sizes controlled within a few thousand items. It is not possible to conduct large data volume testing.

Product introduction: Weka is open source, and the automatic modeling function is an extension module of Weka, which is free to use. Rapidminer is a commercial software. Although it has a free version, the auto model function will be charged.

Overall user experience:Ymodel has the fastest modeling speed. Rapidminer is relatively fast in model building, and when there are many variables, the modeling time increases significantly. Weka modeling requires setting the modeling time beforehand, and the modeling speed is also relatively slow. In Weka, sometimes it is necessary to manually handle some variable types in order to be recognized by automatic modeling. In terms of automatic modeling functionality, Weka's experience is relatively poor.

Testing method:All data is divided into a training set and a prediction set, and the prediction results are exported and scored uniformly.

Test results:

1. Titanic Survival Prediction - Classification

Training data: 802 items, 12 variables

The ratio of positive and negative samples is approximately 3:5

It is unable to output probability values in Weka (or possibly not finding how to output), therefore unable to calculate AUC.

2. House Price Prediction - Regression

3. Credit Company User Overdue Prediction - Classification

Training data: 8938 items, 56 variables

The ratio of positive and negative samples is approximately 1:8

On this data, the Weka model failed and did not capture any positive sample.

4. Claims prediction of insurance company policies - classification

Training data: 3470 items, 29 variables

The ratio of positive and negative samples is approximately 1:7

5. Second-hand car transaction price prediction

Overall evaluation:Among the 5 data samples used in this testing, the rankings vary depending on the data, but the difference in indexes is not significant, and the overall performance of Ymodel is quite good. In comparison, Weka performs well in regression model, Ymodel performs well in classification model, and Rapidminer is in the middle.