西安网站开发外包,电子商务网站分析,信用网站建设工作简报,网站上常用的字体回归分析
回归分析属于监督学习方法的一种#xff0c;主要用于预测连续型目标变量#xff0c;可以预测、计算趋势以及确定变量之间的关系等。
Regession Evaluation Metrics
以下是一些最流行的回归评估指标: 平均绝对误差(MAE):目标变量的预测值与实际值之间的平均绝对差…回归分析
回归分析属于监督学习方法的一种主要用于预测连续型目标变量可以预测、计算趋势以及确定变量之间的关系等。
Regession Evaluation Metrics
以下是一些最流行的回归评估指标: 平均绝对误差(MAE):目标变量的预测值与实际值之间的平均绝对差值。 均方误差(MSE):目标变量的预测值与实际值之间的平均平方差。 均方根误差(RMSE):均方根误差的平方根。 Huber Loss:一种混合损失函数在较大误差时从MAE过渡到MSE在鲁棒性和MSE对异常值的敏感性之间提供平衡。 均方根对数误差 R2-Score
分类模型
决策树(监督分类回归模型)
分类树:该树用于确定目标变量在连续时最有可能落入哪个“类”。 回归树:用于预测连续变量的值。 在决策树中节点根据属性的阈值划分为子节点。将根节点作为训练集并根据最优属性和阈值将其分割为两个节点。此外子集也使用相同的逻辑进行分割。这个过程一直持续直到在树中找到最后一个纯子集或者在该生长的树中找到最大可能的叶子数。
根据分割指标和分割方法可分为ID3、C4.5、CART算法。 1ID3算法以信息增益为准则来选择最优划分属性 信息增益的计算是基于信息熵度量样本集合纯度的指标 2C4.5基于信息增益率准则 选择最有分割属性的算法 3. CART:以基尼系数为准则选择最优划分属性可用于分类和回归 基尼杂质-基尼杂质测量根据多数类标记的子集对随机实例进行错误分类的概率。基尼不纯系数越低意味着子集的纯度越高。分割标准- CART算法评估每个节点上的所有潜在分割并选择最能减少结果子集的基尼杂质的分割。这个过程一直持续直到达到一个停止条件比如最大树深度或叶子节点中的最小实例数。 sklearn.tree.DecisionTreeClassifier分类
class sklearn.tree.DecisionTreeClassifier(*, criteriongini, splitterbest, max_depthNone, min_samples_split2, min_samples_leaf1, min_weight_fraction_leaf0.0, max_featuresNone, random_stateNone, max_leaf_nodesNone, min_impurity_decrease0.0, class_weightNone, ccp_alpha0.0)[source]from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
cancer load_breast_cancer()
X_train, X_test, y_train, y_test train_test_split(cancer.data, cancer.target, stratifycancer.target, random_state42)
tree DecisionTreeClassifier(random_state0)
tree.fit(X_train, y_train)
print(Accuracy on training set: {:.3f}.format(tree.score(X_train, y_train)))
print(Accuracy on test set: {:.3f}.format(tree.score(X_test, y_test)))tree DecisionTreeClassifier(max_depth4, random_state0)
tree.fit(X_train, y_train)print(Accuracy on training set: {:.3f}.format(tree.score(X_train, y_train)))
print(Accuracy on test set: {:.3f}.format(tree.score(X_test, y_test)))fig, axes plt.subplots(2, 3, figsize(20, 10))
for i, (ax, tree) in enumerate(zip(axes.ravel(), forest.estimators_)):ax.set_title(Tree {}.format(i))mglearn.plots.plot_tree_partition(X_train, y_train, tree, axax)mglearn.plots.plot_2d_separator(forest, X_train, fillTrue, axaxes[-1, -1],alpha.4)
axes[-1, -1].set_title(Random Forest)
mglearn.discrete_scatter(X_train[:, 0], X_train[:, 1], y_train)def plot_feature_importances_cancer(model):n_features cancer.data.shape[1]plt.barh(np.arange(n_features), model.feature_importances_, aligncenter)plt.yticks(np.arange(n_features), cancer.feature_names)plt.xlabel(Feature importance)plt.ylabel(Feature)plt.ylim(-1, n_features)plot_feature_importances_cancer(tree)from sklearn.tree import DecisionTreeClassifier
from sklearn.preprocessing import LabelEncoder# Define the features and target variable
features [[red, large],[green, small],[red, small],[yellow, large],[green, large],[orange, large],
]
target_variable [apple, lime, strawberry, banana, grape, orange]# Flatten the features list for encoding
flattened_features [item for sublist in features for item in sublist]# Use a single LabelEncoder for all features and target variable
le LabelEncoder()
le.fit(flattened_features target_variable)# Encode features and target variable
encoded_features [le.transform(item) for item in features]
encoded_target le.transform(target_variable)# Create a CART classifier
clf DecisionTreeClassifier()# Train the classifier on the training set
clf.fit(encoded_features, encoded_target)# Predict the fruit type for a new instance
new_instance [red, large]
encoded_new_instance le.transform(new_instance)
predicted_fruit_type clf.predict([encoded_new_instance])
decoded_predicted_fruit_type le.inverse_transform(predicted_fruit_type)
print(Predicted fruit type:, decoded_predicted_fruit_type[0])DecisionTreeRegressor回归
import os
ram_prices pd.read_csv(os.path.join(mglearn.datasets.DATA_PATH, ram_price.csv))plt.semilogy(ram_prices.date, ram_prices.price)
plt.xlabel(Year)
plt.ylabel(Price in $/Mbyte)from sklearn.tree import DecisionTreeRegressor
# use historical data to forecast prices after the year 2000
data_train ram_prices[ram_prices.date 2000]
data_test ram_prices[ram_prices.date 2000]# predict prices based on date
X_train data_train.date[:, np.newaxis]
# we use a log-transform to get a simpler relationship of data to target
y_train np.log(data_train.price)tree DecisionTreeRegressor(max_depth3).fit(X_train, y_train)
linear_reg LinearRegression().fit(X_train, y_train)# predict on all data
X_all ram_prices.date[:, np.newaxis]pred_tree tree.predict(X_all)
pred_lr linear_reg.predict(X_all)# undo log-transform
price_tree np.exp(pred_tree)
price_lr np.exp(pred_lr)plt.semilogy(data_train.date, data_train.price, labelTraining data)
plt.semilogy(data_test.date, data_test.price, labelTest data)
plt.semilogy(ram_prices.date, price_tree, labelTree prediction)
plt.semilogy(ram_prices.date, price_lr, labelLinear prediction)
plt.legend()随机森林集成学习
先补充组合分类器的概念将多个分类器的结果进行多票表决或取平均值以此作为最终的结果。 每个决策树都有很高的方差但是当我们将它们并行地组合在一起时结果的方差就会很低因为每个决策树都在特定的样本数据上得到了完美的训练因此输出不依赖于一个决策树而是依赖于多个决策树。在分类问题的情况下使用多数投票分类器获得最终输出。在回归问题的情况下最终输出是所有输出的平均值。这部分称为聚合。 1.构建组合分类器的好处 (1)提升模型精度整合各个模型的分类结果得到更合理的决策边界减少整体错误呢实现更好的分类效果 (2)处理过大或过小的数据集数据集较大时可将数据集划分成多个子集对子集构建分类器当数据集较小时通过自助采样(bootstrap)从原始数据集采样产生多组不同的数据集构建分类器。
(3)若决策边界过于复杂则线性模型不能很好地描述真实情况。因此现对于特定区域的数据集训练多个线性分类器再将他们集成。 (4)比较适合处理多源异构数据存储方式不同关系型、非关系型类别不同时序型、离散型、连续型、网络结构数据
随机森林是一个多决策树的组合分类器随机主要体现在两个方面数据选取的随机性和特征选取的随机性。
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_moonsX, y make_moons(n_samples100, noise0.25, random_state3)
X_train, X_test, y_train, y_test train_test_split(X, y, stratifyy,random_state42)forest RandomForestClassifier(n_estimators5, random_state2)
forest.fit(X_train, y_train)fig, axes plt.subplots(2, 3, figsize(20, 10))
for i, (ax, tree) in enumerate(zip(axes.ravel(), forest.estimators_)):ax.set_title(Tree {}.format(i))mglearn.plots.plot_tree_partition(X_train, y_train, tree, axax)mglearn.plots.plot_2d_separator(forest, X_train, fillTrue, axaxes[-1, -1],alpha.4)
axes[-1, -1].set_title(Random Forest)
mglearn.discrete_scatter(X_train[:, 0], X_train[:, 1], y_train)X_train, X_test, y_train, y_test train_test_split(cancer.data, cancer.target, random_state0)
forest RandomForestClassifier(n_estimators100, random_state0)
forest.fit(X_train, y_train)print(Accuracy on training set: {:.3f}.format(forest.score(X_train, y_train)))
print(Accuracy on test set: {:.3f}.format(forest.score(X_test, y_test)))plot_feature_importances_cancer(forest)我们举一个线性回归的例子。我们有一个住房数据集我们想预测房子的价格。下面是它的python代码。
# Python code to illustrate
# regression using data set
import matplotlib
matplotlib.use(GTKAgg)import matplotlib.pyplot as plt
import numpy as np
from sklearn import datasets, linear_model
import pandas as pd# Load CSV and columns
df pd.read_csv(Housing.csv)Y df[price]
X df[lotsize]XX.values.reshape(len(X),1)
YY.values.reshape(len(Y),1)# Split the data into training/testing sets
X_train X[:-250]
X_test X[-250:]# Split the targets into training/testing sets
Y_train Y[:-250]
Y_test Y[-250:]# Plot outputs
plt.scatter(X_test, Y_test, colorblack)
plt.title(Test Data)
plt.xlabel(Size)
plt.ylabel(Price)
plt.xticks(())
plt.yticks(())
# Create linear regression object
regr linear_model.LinearRegression()# Train the model using the training sets
regr.fit(X_train, Y_train)# Plot outputs
plt.plot(X_test, regr.predict(X_test), colorred,linewidth3)
plt.show()在这张图中我们绘制了测试数据。红线表示预测价格的最佳拟合线。使用线性回归模型进行个体预测: print( str(round(regr.predict(5000))) )
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import sklearn
import warningsfrom sklearn.preprocessing import LabelEncoder
from sklearn.impute import KNNImputer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import f1_score
from sklearn.ensemble import RandomForestRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import cross_val_scorewarnings.filterwarnings(ignore)df pd.read_csv(Salaries.csv)
print(df)Here the .info() method provides a quick overview of the structure, data types, and memory usage of the dataset.
df.info()# Assuming df is your DataFrame
X df.iloc[:,1:2].values #features
y df.iloc[:,2].values # Target variablestep 4: Random Forest Regressor model代码对分类数据进行数字编码处理将处理后的数据与数字数据结合起来使用准备好的数据训练Random Forest Regression模型。
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import LabelEncoderCheck for and handle categorical variables
label_encoder LabelEncoder()
x_categorical df.select_dtypes(include[object]).apply(label_encoder.fit_transform)
x_numerical df.select_dtypes(exclude[object]).values
x pd.concat([pd.DataFrame(x_numerical), x_categorical], axis1).values# Fitting Random Forest Regression to the dataset
regressor RandomForestRegressor(n_estimators10, random_state0, oob_scoreTrue)# Fit the regressor with x and y data
regressor.fit(x, y)# Evaluating the model
from sklearn.metrics import mean_squared_error, r2_score# Access the OOB Score
oob_score regressor.oob_score_
print(fOut-of-Bag Score: {oob_score})# Making predictions on the same data or new data
predictions regressor.predict(x)# Evaluating the model
mse mean_squared_error(y, predictions)
print(fMean Squared Error: {mse})r2 r2_score(y, predictions)
print(fR-squared: {r2})import numpy as np
X_grid np.arange(min(X),max(X),0.01)
X_grid X_grid.reshape(len(X_grid),1) plt.scatter(X,y, colorblue) #plotting real points
plt.plot(X_grid, regressor.predict(X_grid),colorgreen) #plotting for predict pointsplt.title(Random Forest Regression Results)
plt.xlabel(Position level)
plt.ylabel(Salary)
plt.show()from sklearn.tree import plot_tree
import matplotlib.pyplot as plt# Assuming regressor is your trained Random Forest model
# Pick one tree from the forest, e.g., the first tree (index 0)
tree_to_plot regressor.estimators_[0]# Plot the decision tree
plt.figure(figsize(20, 10))
plot_tree(tree_to_plot, feature_namesdf.columns.tolist(), filledTrue, roundedTrue, fontsize10)
plt.title(Decision Tree from Random Forest)
plt.show()
