msb-public-java
/
AiLearning-Theory-Applying
mirror of https://github.com/ben1234560/AiLearning-Theory-Applying
1
4
Code Issues Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '95bff73b15'
${ noResults }
AiLearning-Theory-Applying/竞赛优胜技巧/Optuna一种超参数优化框架.ipynb

4372 lines
331 KiB
Raw Blame History Escape

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

{
"cells": [
{
"cell_type": "markdown",
"id": "b3d4d608",
"metadata": {},
"source": [
"# Optuna一种超参数优化框架\n",
"https://github.com/optuna/optuna\n",
"\n",
"分别采用两组数据集进行比较，分别是加利福尼亚住房数据集（回归）和森林植被类型（多分类）"
]
},
{
"cell_type": "markdown",
"id": "b22b6f90",
"metadata": {},
"source": [
"## 结论：效果不错"
]
},
{
"cell_type": "markdown",
"id": "a2d41e62",
"metadata": {},
"source": [
"## 波士顿房价预测任务（回归）"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f5e0e977",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import time\n",
"import gc\n",
"\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.metrics import mean_squared_error\n",
"import lightgbm as lgb # 使用lgb模型"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5eda2637",
"metadata": {},
"outputs": [],
"source": [
"from sklearn.datasets import fetch_california_housing\n",
"data = fetch_california_housing()\n",
"X, y = data['data'], data['target']"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "722beb8e",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>MedInc</th>\n",
" <th>HouseAge</th>\n",
" <th>AveRooms</th>\n",
" <th>AveBedrms</th>\n",
" <th>Population</th>\n",
" <th>AveOccup</th>\n",
" <th>Latitude</th>\n",
" <th>Longitude</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>8.3252</td>\n",
" <td>41.0</td>\n",
" <td>6.984127</td>\n",
" <td>1.02381</td>\n",
" <td>322.0</td>\n",
" <td>2.555556</td>\n",
" <td>37.88</td>\n",
" <td>-122.23</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>8.3014</td>\n",
" <td>21.0</td>\n",
" <td>6.238137</td>\n",
" <td>0.97188</td>\n",
" <td>2401.0</td>\n",
" <td>2.109842</td>\n",
" <td>37.86</td>\n",
" <td>-122.22</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude \\\n",
"0 8.3252 41.0 6.984127 1.02381 322.0 2.555556 37.88 \n",
"1 8.3014 21.0 6.238137 0.97188 2401.0 2.109842 37.86 \n",
"\n",
" Longitude \n",
"0 -122.23 \n",
"1 -122.22 "
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X = pd.DataFrame(X,columns=data.feature_names)\n",
"X.head(2)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "08ebab89",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"RangeIndex: 20640 entries, 0 to 20639\n",
"Data columns (total 8 columns):\n",
" # Column Non-Null Count Dtype \n",
"--- ------ -------------- ----- \n",
" 0 MedInc 20640 non-null float64\n",
" 1 HouseAge 20640 non-null float64\n",
" 2 AveRooms 20640 non-null float64\n",
" 3 AveBedrms 20640 non-null float64\n",
" 4 Population 20640 non-null float64\n",
" 5 AveOccup 20640 non-null float64\n",
" 6 Latitude 20640 non-null float64\n",
" 7 Longitude 20640 non-null float64\n",
"dtypes: float64(8)\n",
"memory usage: 1.3 MB\n"
]
}
],
"source": [
"X.info()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8932c66c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"训练集： (18576, 8)\n",
"测试集： (2064, 8)\n"
]
}
],
"source": [
"# 切分训练和测试集\n",
"train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n",
"train_y = pd.DataFrame(train_y,columns=['result'])\n",
"test_y = pd.DataFrame(test_y,columns=['result'])\n",
"\n",
"print('训练集：',train_x.shape)\n",
"print('测试集：', test_x.shape)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "94561283",
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import KFold\n",
"skf = KFold(n_splits=5, shuffle=True, random_state=42)"
]
},
{
"cell_type": "markdown",
"id": "e87a2121",
"metadata": {},
"source": [
"### 使用LGB作为模型并CV，不使用optuna调参"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "5cad8967",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000866 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1837\n",
"[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.073247\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.492877\n",
"[100]\tvalid_0's rmse: 0.469235\n",
"[150]\tvalid_0's rmse: 0.460357\n",
"[200]\tvalid_0's rmse: 0.455302\n",
"[250]\tvalid_0's rmse: 0.452674\n",
"[300]\tvalid_0's rmse: 0.451179\n",
"Early stopping, best iteration is:\n",
"[313]\tvalid_0's rmse: 0.450303\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000494 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.070396\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.496484\n",
"[100]\tvalid_0's rmse: 0.476225\n",
"[150]\tvalid_0's rmse: 0.469582\n",
"[200]\tvalid_0's rmse: 0.464643\n",
"[250]\tvalid_0's rmse: 0.463244\n",
"[300]\tvalid_0's rmse: 0.461886\n",
"[350]\tvalid_0's rmse: 0.460305\n",
"[400]\tvalid_0's rmse: 0.4596\n",
"[450]\tvalid_0's rmse: 0.458645\n",
"Early stopping, best iteration is:\n",
"[435]\tvalid_0's rmse: 0.458368\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000800 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1837\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.069154\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.48676\n",
"[100]\tvalid_0's rmse: 0.466911\n",
"[150]\tvalid_0's rmse: 0.459693\n",
"[200]\tvalid_0's rmse: 0.457631\n",
"[250]\tvalid_0's rmse: 0.452574\n",
"Early stopping, best iteration is:\n",
"[264]\tvalid_0's rmse: 0.451947\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000779 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.063366\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.492482\n",
"[100]\tvalid_0's rmse: 0.468929\n",
"[150]\tvalid_0's rmse: 0.459713\n",
"[200]\tvalid_0's rmse: 0.456203\n",
"[250]\tvalid_0's rmse: 0.454063\n",
"[300]\tvalid_0's rmse: 0.451901\n",
"Early stopping, best iteration is:\n",
"[322]\tvalid_0's rmse: 0.450961\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000694 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.066924\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.488385\n",
"[100]\tvalid_0's rmse: 0.464851\n",
"[150]\tvalid_0's rmse: 0.455183\n",
"[200]\tvalid_0's rmse: 0.450506\n",
"[250]\tvalid_0's rmse: 0.4461\n",
"[300]\tvalid_0's rmse: 0.44438\n",
"[350]\tvalid_0's rmse: 0.441808\n",
"[400]\tvalid_0's rmse: 0.44079\n",
"Early stopping, best iteration is:\n",
"[411]\tvalid_0's rmse: 0.440543\n",
"Wall time: 3.44 s\n"
]
}
],
"source": [
"%%time\n",
"test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n",
"params = {'boosting_type': 'gbdt',\n",
" 'objective': 'regression',\n",
" \"metric\": 'rmse'}\n",
"for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
" dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
" dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
" model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
" verbose_eval=50,\n",
" early_stopping_rounds=20,\n",
" num_boost_round=5000)\n",
" test_predict[:,i] = model.predict(test_x)\n",
"predict = np.mean(test_predict,axis=1)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "886bfdad",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.4346521330333544"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 评估指标rmse，越小越好\n",
"np.sqrt(mean_squared_error(test_y,predict))"
]
},
{
"cell_type": "markdown",
"id": "c271442c",
"metadata": {},
"source": [
"### 使用LGB作为模型并CV，使用optuna调参"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "9a87f9db",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# pip install optuna"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "d8a0279a",
"metadata": {},
"outputs": [],
"source": [
"import optuna"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "7433fdcb",
"metadata": {},
"outputs": [],
"source": [
"def objective(trial,train_x, train_y, valid_x, valid_y):\n",
" dtrain = lgb.Dataset(train_x, label=train_y)\n",
" dvalid = lgb.Dataset(valid_x, label=valid_y)\n",
"\n",
" param = {\n",
" \"objective\": \"regression\",\n",
" \"metric\": \"rmse\",\n",
" \"verbosity\": -1,\n",
" \"boosting_type\": \"gbdt\",\n",
" 'random_state':42,\n",
" \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n",
" \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n",
" \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n",
" \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n",
" \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n",
" \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n",
" \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n",
" }\n",
"\n",
" # Add a callback for pruning.\n",
" pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"rmse\")\n",
" gbm = lgb.train(\n",
" param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n",
" )\n",
"\n",
" preds = gbm.predict(valid_x)\n",
" pred_labels = np.rint(preds)\n",
" rmse = np.sqrt(mean_squared_error(valid_y,pred_labels))\n",
" return rmse"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "b9018adb",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:27:35,965]\u001b[0m A new study created in memory with name: no-name-cc424a48-83c6-4329-92a3-261d08a5edb9\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:36,779]\u001b[0m Trial 0 finished with value: 0.5178928385298298 and parameters: {'lambda_l1': 0.9464949700025425, 'lambda_l2': 1.4692321446797693e-05, 'num_leaves': 135, 'feature_fraction': 0.6448685329355106, 'bagging_fraction': 0.9048970068857253, 'bagging_freq': 7, 'min_child_samples': 30}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:37,641]\u001b[0m Trial 1 finished with value: 0.5336017055299863 and parameters: {'lambda_l1': 0.6354399847075458, 'lambda_l2': 4.87700419620481, 'num_leaves': 237, 'feature_fraction': 0.895991820221385, 'bagging_fraction': 0.6704364287317786, 'bagging_freq': 3, 'min_child_samples': 49}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:38,288]\u001b[0m Trial 2 finished with value: 0.5345721353318472 and parameters: {'lambda_l1': 9.52099217128906e-05, 'lambda_l2': 1.1454590646374039e-07, 'num_leaves': 170, 'feature_fraction': 0.7899599630691868, 'bagging_fraction': 0.5059741439893897, 'bagging_freq': 2, 'min_child_samples': 50}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:38,956]\u001b[0m Trial 3 finished with value: 0.543693048566443 and parameters: {'lambda_l1': 0.1481577233973877, 'lambda_l2': 3.179336503731134e-07, 'num_leaves': 128, 'feature_fraction': 0.9783519466402901, 'bagging_fraction': 0.6889127950806107, 'bagging_freq': 5, 'min_child_samples': 37}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:39,424]\u001b[0m Trial 4 finished with value: 0.5501760993688642 and parameters: {'lambda_l1': 3.7638689664291936, 'lambda_l2': 0.020384770050374573, 'num_leaves': 175, 'feature_fraction': 0.4840841073088404, 'bagging_fraction': 0.8334033110992205, 'bagging_freq': 5, 'min_child_samples': 35}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:39,493]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:39,774]\u001b[0m Trial 6 pruned. Trial was pruned at iteration 33.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:39,828]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:40,187]\u001b[0m Trial 8 finished with value: 0.5249490156742078 and parameters: {'lambda_l1': 8.186728837628555e-05, 'lambda_l2': 2.6479736415938615, 'num_leaves': 72, 'feature_fraction': 0.7599981859922114, 'bagging_fraction': 0.9692259725506919, 'bagging_freq': 5, 'min_child_samples': 38}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:40,299]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:40,353]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:40,398]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:40,470]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:41,280]\u001b[0m Trial 13 finished with value: 0.5325142394678348 and parameters: {'lambda_l1': 0.009551573559223583, 'lambda_l2': 0.0003114297759461472, 'num_leaves': 152, 'feature_fraction': 0.8236610445865283, 'bagging_fraction': 0.8389170404125451, 'bagging_freq': 4, 'min_child_samples': 20}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:41,391]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,015]\u001b[0m Trial 15 finished with value: 0.5314302398651312 and parameters: {'lambda_l1': 0.024227381422510946, 'lambda_l2': 0.33276527010426776, 'num_leaves': 91, 'feature_fraction': 0.7874026441153832, 'bagging_fraction': 0.7899555875604796, 'bagging_freq': 7, 'min_child_samples': 19}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,091]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,154]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,226]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,338]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,384]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,829]\u001b[0m Trial 21 finished with value: 0.5245233417218278 and parameters: {'lambda_l1': 0.07296901453015396, 'lambda_l2': 0.19272462963641587, 'num_leaves': 90, 'feature_fraction': 0.752955332537125, 'bagging_fraction': 0.7944607588474272, 'bagging_freq': 7, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:42,901]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:43,287]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 97.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:43,371]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:43,489]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:43,572]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:43,632]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:44,171]\u001b[0m Trial 28 finished with value: 0.5255422897694476 and parameters: {'lambda_l1': 0.028260092955639447, 'lambda_l2': 0.08413723627697263, 'num_leaves': 74, 'feature_fraction': 0.8042469147694201, 'bagging_fraction': 0.7548217217698655, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:44,331]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:44,516]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:44,637]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:44,723]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:45,592]\u001b[0m Trial 33 finished with value: 0.5218146276752049 and parameters: {'lambda_l1': 0.0016497886750146257, 'lambda_l2': 0.21484669274687151, 'num_leaves': 133, 'feature_fraction': 0.6885034648040802, 'bagging_fraction': 0.9281320338482011, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:45,711]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:46,669]\u001b[0m Trial 35 finished with value: 0.5205542553680208 and parameters: {'lambda_l1': 4.3493248665689235e-05, 'lambda_l2': 3.680853885692289, 'num_leaves': 169, 'feature_fraction': 0.7104567429350184, 'bagging_fraction': 0.8672533271233644, 'bagging_freq': 1, 'min_child_samples': 23}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:47,676]\u001b[0m Trial 36 finished with value: 0.5206023202395521 and parameters: {'lambda_l1': 0.0006187306261428898, 'lambda_l2': 6.130428276943865e-08, 'num_leaves': 168, 'feature_fraction': 0.7036284508576037, 'bagging_fraction': 0.868428878180794, 'bagging_freq': 2, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:47,808]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:47,937]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:48,844]\u001b[0m Trial 39 finished with value: 0.5191042263340773 and parameters: {'lambda_l1': 0.0005562321404093257, 'lambda_l2': 2.232696278872472e-08, 'num_leaves': 161, 'feature_fraction': 0.710527336424428, 'bagging_fraction': 0.9309118257124477, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:49,889]\u001b[0m Trial 40 finished with value: 0.5168505162425636 and parameters: {'lambda_l1': 0.0004441190594738051, 'lambda_l2': 1.0707946566246148e-08, 'num_leaves': 190, 'feature_fraction': 0.7132737394337006, 'bagging_fraction': 0.8296082050311246, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:50,929]\u001b[0m Trial 41 finished with value: 0.5262832333602245 and parameters: {'lambda_l1': 0.0004892025280466791, 'lambda_l2': 1.89573604152611e-08, 'num_leaves': 189, 'feature_fraction': 0.7134600565845959, 'bagging_fraction': 0.8334328552244646, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:27:51,023]\u001b[0m Trial 42 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:51,805]\u001b[0m Trial 43 finished with value: 0.516065766031832 and parameters: {'lambda_l1': 0.00021869661941011055, 'lambda_l2': 1.0674672013946999e-08, 'num_leaves': 201, 'feature_fraction': 0.7039965594752932, 'bagging_fraction': 0.8165783106186105, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:51,940]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:52,044]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:52,218]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:53,067]\u001b[0m Trial 47 finished with value: 0.5256815758791472 and parameters: {'lambda_l1': 6.162602167779085e-05, 'lambda_l2': 1.6462819728079273e-07, 'num_leaves': 201, 'feature_fraction': 0.7163097853297453, 'bagging_fraction': 0.9420618119436382, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:53,178]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:53,762]\u001b[0m Trial 49 finished with value: 0.5213791636070063 and parameters: {'lambda_l1': 2.774750232046531e-07, 'lambda_l2': 2.4404405074646332e-08, 'num_leaves': 154, 'feature_fraction': 0.6882189607163384, 'bagging_fraction': 0.9871043449165129, 'bagging_freq': 1, 'min_child_samples': 9}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:53,875]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:54,587]\u001b[0m Trial 51 finished with value: 0.5174618905113623 and parameters: {'lambda_l1': 0.000381507052985562, 'lambda_l2': 4.901891994722806e-08, 'num_leaves': 166, 'feature_fraction': 0.7117978094083592, 'bagging_fraction': 0.8956613846537137, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:55,304]\u001b[0m Trial 52 finished with value: 0.5219780933938188 and parameters: {'lambda_l1': 0.0003055316771241415, 'lambda_l2': 1.0975399369852818e-08, 'num_leaves': 175, 'feature_fraction': 0.7709159871191618, 'bagging_fraction': 0.8946659725371712, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:56,481]\u001b[0m Trial 53 finished with value: 0.522019841081264 and parameters: {'lambda_l1': 3.0853077659712017e-06, 'lambda_l2': 3.167435996190143e-08, 'num_leaves': 199, 'feature_fraction': 0.700370780125537, 'bagging_fraction': 0.9659440325481246, 'bagging_freq': 1, 'min_child_samples': 5}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:57,422]\u001b[0m Trial 54 finished with value: 0.5210662969810745 and parameters: {'lambda_l1': 7.857450656938453e-05, 'lambda_l2': 4.5159973730477044e-07, 'num_leaves': 153, 'feature_fraction': 0.7263108852936352, 'bagging_fraction': 0.936630215722916, 'bagging_freq': 1, 'min_child_samples': 19}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:57,568]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:57,678]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:57,803]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:57,889]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:58,284]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:58,510]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 15.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:59,683]\u001b[0m Trial 61 finished with value: 0.5238780309760502 and parameters: {'lambda_l1': 0.0005622649168938934, 'lambda_l2': 7.041773458860283e-08, 'num_leaves': 165, 'feature_fraction': 0.6971774484774765, 'bagging_fraction': 0.8845735572162268, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:27:59,867]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:01,236]\u001b[0m Trial 63 finished with value: 0.5172018048571797 and parameters: {'lambda_l1': 4.4774427854289624e-05, 'lambda_l2': 1.0431434685856767e-07, 'num_leaves': 175, 'feature_fraction': 0.7067550902029666, 'bagging_fraction': 0.7960327498051027, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:01,369]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:01,513]\u001b[0m Trial 65 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:02,393]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 76.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:02,529]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:02,721]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:03,060]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 44.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:03,135]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:03,486]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 56.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:03,587]\u001b[0m Trial 72 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,107]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 68.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,203]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,307]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,631]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 39.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,725]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,831]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,920]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:04,992]\u001b[0m Trial 80 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:05,099]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:05,766]\u001b[0m Trial 82 finished with value: 0.5178938881576146 and parameters: {'lambda_l1': 1.2203483524616316e-08, 'lambda_l2': 5.507928324956204e-07, 'num_leaves': 185, 'feature_fraction': 0.7219415365227362, 'bagging_fraction': 0.9438920094595108, 'bagging_freq': 1, 'min_child_samples': 17}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:05,866]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:05,961]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:06,778]\u001b[0m Trial 85 finished with value: 0.5122528701381253 and parameters: {'lambda_l1': 6.371822640459341e-07, 'lambda_l2': 3.03967794068636e-08, 'num_leaves': 189, 'feature_fraction': 0.7066521328434867, 'bagging_fraction': 0.8972190570020028, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:07,709]\u001b[0m Trial 86 finished with value: 0.5128398316765018 and parameters: {'lambda_l1': 1.5059254193489838e-07, 'lambda_l2': 2.6171717314455622e-08, 'num_leaves': 192, 'feature_fraction': 0.711268040370677, 'bagging_fraction': 0.9174432395706226, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:09,411]\u001b[0m Trial 87 finished with value: 0.5191456815789754 and parameters: {'lambda_l1': 5.687878085793806e-08, 'lambda_l2': 2.6772763929664963e-08, 'num_leaves': 216, 'feature_fraction': 0.7413398142797374, 'bagging_fraction': 0.9993988673163303, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:09,813]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:12,526]\u001b[0m Trial 89 finished with value: 0.5102536883069437 and parameters: {'lambda_l1': 1.6446541932580473e-07, 'lambda_l2': 1.0734412329614426e-07, 'num_leaves': 193, 'feature_fraction': 0.8123395009389368, 'bagging_fraction': 0.9184649865458648, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:12,792]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:13,099]\u001b[0m Trial 91 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:13,456]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:13,664]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:14,351]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 45.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:14,508]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:14,828]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:14,962]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:16,197]\u001b[0m Trial 98 finished with value: 0.5171555001746302 and parameters: {'lambda_l1': 1.0103137663726183e-08, 'lambda_l2': 1.7418953701287595e-06, 'num_leaves': 203, 'feature_fraction': 0.7265189263478641, 'bagging_fraction': 0.9317931154547426, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:16,557]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 16.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 100\n",
"Best trial:\n",
" Value: 0.5102536883069437\n",
" Params: \n",
" lambda_l1: 1.6446541932580473e-07\n",
" lambda_l2: 1.0734412329614426e-07\n",
" num_leaves: 193\n",
" feature_fraction: 0.8123395009389368\n",
" bagging_fraction: 0.9184649865458648\n",
" bagging_freq: 1\n",
" min_child_samples: 10\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000712 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1837\n",
"[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.073247\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.453164\n",
"[100]\tvalid_0's rmse: 0.442958\n",
"Early stopping, best iteration is:\n",
"[110]\tvalid_0's rmse: 0.442157\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:17,677]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:17,796]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:17,941]\u001b[0m Trial 102 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,090]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,293]\u001b[0m Trial 104 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,412]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,524]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,647]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,797]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:18,911]\u001b[0m Trial 109 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:19,029]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:19,211]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:19,595]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 30.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:19,744]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:19,853]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,146]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 23.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,267]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,420]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,594]\u001b[0m Trial 118 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,780]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:20,942]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:21,073]\u001b[0m Trial 121 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:21,410]\u001b[0m Trial 122 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:21,601]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:21,799]\u001b[0m Trial 124 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:21,944]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,080]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,209]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,367]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,509]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,673]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,754]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:22,923]\u001b[0m Trial 132 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:23,074]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:23,442]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:23,682]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:23,824]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:24,010]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:24,160]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:24,458]\u001b[0m Trial 139 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:24,678]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:24,849]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:25,081]\u001b[0m Trial 142 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:25,256]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:25,501]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:26,208]\u001b[0m Trial 145 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:26,709]\u001b[0m Trial 146 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:26,919]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:27,534]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:27,729]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,152]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 24.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,278]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,393]\u001b[0m Trial 152 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,543]\u001b[0m Trial 153 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,653]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,757]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:28,887]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,001]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,176]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,332]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,541]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,732]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,884]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:29,997]\u001b[0m Trial 163 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,103]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,225]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,330]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,487]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,579]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,685]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,798]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,902]\u001b[0m Trial 171 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:30,999]\u001b[0m Trial 172 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,099]\u001b[0m Trial 173 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,206]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,352]\u001b[0m Trial 175 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,476]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,716]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,834]\u001b[0m Trial 178 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:31,983]\u001b[0m Trial 179 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,099]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,272]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,431]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,546]\u001b[0m Trial 183 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,656]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,763]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:32,965]\u001b[0m Trial 186 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:33,183]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:33,409]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:33,571]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:33,719]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:33,919]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:34,112]\u001b[0m Trial 192 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:34,276]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:34,426]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:34,633]\u001b[0m Trial 195 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:34,781]\u001b[0m Trial 196 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:35,002]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:35,242]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:35,408]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 200\n",
"Best trial:\n",
" Value: 0.5102536883069437\n",
" Params: \n",
" lambda_l1: 1.6446541932580473e-07\n",
" lambda_l2: 1.0734412329614426e-07\n",
" num_leaves: 193\n",
" feature_fraction: 0.8123395009389368\n",
" bagging_fraction: 0.9184649865458648\n",
" bagging_freq: 1\n",
" min_child_samples: 10\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000777 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.070396\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.465875\n",
"Early stopping, best iteration is:\n",
"[76]\tvalid_0's rmse: 0.460117\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:36,810]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:36,973]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,136]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,255]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,402]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,545]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,742]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:37,864]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,005]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,155]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,307]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,419]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,537]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,637]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,786]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:38,962]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,091]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,246]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,368]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,505]\u001b[0m Trial 219 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,637]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,737]\u001b[0m Trial 221 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,847]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:39,948]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,046]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,138]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,235]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,324]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,398]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,483]\u001b[0m Trial 229 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,621]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,779]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,860]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:40,937]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,077]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,188]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,312]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,431]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,533]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,664]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,790]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:41,902]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,009]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,110]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,233]\u001b[0m Trial 244 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,443]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,587]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,674]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,779]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,859]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:42,958]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,070]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,257]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,356]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,462]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,566]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,627]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,743]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,856]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:43,945]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,059]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,164]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,279]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,377]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,478]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,585]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,688]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,794]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:44,914]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,016]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,111]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,248]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,424]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,578]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,809]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:45,937]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,068]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,210]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,357]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,505]\u001b[0m Trial 279 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,662]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,828]\u001b[0m Trial 281 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:46,927]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:47,157]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:47,394]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:47,564]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:47,701]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:47,936]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:48,091]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:48,268]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:48,470]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:48,650]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:48,799]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:48,953]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,112]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,267]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,454]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,631]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,744]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:49,933]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 300\n",
"Best trial:\n",
" Value: 0.5102536883069437\n",
" Params: \n",
" lambda_l1: 1.6446541932580473e-07\n",
" lambda_l2: 1.0734412329614426e-07\n",
" num_leaves: 193\n",
" feature_fraction: 0.8123395009389368\n",
" bagging_fraction: 0.9184649865458648\n",
" bagging_freq: 1\n",
" min_child_samples: 10\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000797 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1837\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.069154\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.451573\n",
"Early stopping, best iteration is:\n",
"[63]\tvalid_0's rmse: 0.44707\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:28:51,004]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,159]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,315]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,412]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,543]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,686]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,842]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:51,977]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,105]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,202]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,341]\u001b[0m Trial 310 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,426]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,642]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,755]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,880]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:52,980]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,093]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,202]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,333]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,488]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,634]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,749]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,863]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:53,983]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,088]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,198]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,321]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,417]\u001b[0m Trial 328 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,486]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,630]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,711]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,838]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:54,968]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,077]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,179]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,319]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,518]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,609]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,741]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:55,903]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,024]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,140]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,425]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,577]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,697]\u001b[0m Trial 345 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,814]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:56,931]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,047]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,165]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,318]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,470]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,640]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,826]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:57,991]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:58,135]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:58,300]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:58,479]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:58,638]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:58,813]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,027]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,183]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,346]\u001b[0m Trial 362 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,510]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,627]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,792]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:28:59,954]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:00,188]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:00,393]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:00,557]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:00,707]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:00,836]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,018]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,171]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,355]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,634]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,803]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:01,949]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,110]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,225]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,409]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,574]\u001b[0m Trial 381 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,733]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:02,895]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:03,059]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:03,184]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:03,408]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:03,583]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:03,729]\u001b[0m Trial 388 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:29:03,924]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,120]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,292]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,406]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,514]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,671]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,760]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:04,879]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:05,049]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:05,188]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:05,277]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 400\n",
"Best trial:\n",
" Value: 0.5102536883069437\n",
" Params: \n",
" lambda_l1: 1.6446541932580473e-07\n",
" lambda_l2: 1.0734412329614426e-07\n",
" num_leaves: 193\n",
" feature_fraction: 0.8123395009389368\n",
" bagging_fraction: 0.9184649865458648\n",
" bagging_freq: 1\n",
" min_child_samples: 10\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000424 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.063366\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.458105\n",
"[100]\tvalid_0's rmse: 0.449598\n",
"[150]\tvalid_0's rmse: 0.449092\n",
"Early stopping, best iteration is:\n",
"[138]\tvalid_0's rmse: 0.448682\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:29:06,435]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:06,586]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:06,756]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:06,867]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:06,968]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,051]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,151]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,290]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,400]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,530]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,649]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,752]\u001b[0m Trial 411 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:07,880]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,002]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,140]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,264]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,434]\u001b[0m Trial 416 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,553]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,677]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,794]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:08,902]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,060]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,181]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,279]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,517]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,691]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:09,858]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,005]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,197]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,377]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,567]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,749]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:10,908]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,012]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,264]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,460]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,636]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,805]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:11,985]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,116]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,257]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,455]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,626]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,785]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:12,916]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,059]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,202]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,402]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,571]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,751]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:13,967]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:14,171]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:14,354]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:14,580]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:14,763]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:14,944]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:15,124]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:15,326]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:15,508]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:15,707]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:15,875]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,043]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,205]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,355]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,499]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,601]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,757]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:16,899]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,040]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,256]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,440]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,613]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,735]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,851]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:17,940]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,015]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,140]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,318]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,485]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,594]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,767]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:18,864]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,000]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,141]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,241]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,362]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,512]\u001b[0m Trial 486 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,651]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:19,777]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 16:29:19,892]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,026]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,152]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,291]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,385]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,508]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,644]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,766]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:20,912]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:21,093]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 16:29:21,228]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 500\n",
"Best trial:\n",
" Value: 0.5102536883069437\n",
" Params: \n",
" lambda_l1: 1.6446541932580473e-07\n",
" lambda_l2: 1.0734412329614426e-07\n",
" num_leaves: 193\n",
" feature_fraction: 0.8123395009389368\n",
" bagging_fraction: 0.9184649865458648\n",
" bagging_freq: 1\n",
" min_child_samples: 10\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.001110 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 1838\n",
"[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
"[LightGBM] [Info] Start training from score 2.066924\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's rmse: 0.444941\n",
"[100]\tvalid_0's rmse: 0.437147\n",
"Early stopping, best iteration is:\n",
"[90]\tvalid_0's rmse: 0.43678\n",
"Wall time: 1min 46s\n"
]
}
],
"source": [
"%%time\n",
"if __name__ == \"__main__\":\n",
" test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n",
" study = optuna.create_study(\n",
" pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\" # 指定是越小越好\n",
" )\n",
" for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
" study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n",
" train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n",
"\n",
" print(\"Number of finished trials: {}\".format(len(study.trials)))\n",
"\n",
" print(\"Best trial:\")\n",
" trial = study.best_trial\n",
"\n",
" print(\" Value: {}\".format(trial.value))\n",
"\n",
" print(\" Params: \")\n",
" for key, value in trial.params.items():\n",
" print(\" {}: {}\".format(key, value))\n",
" \n",
" params = {'boosting_type': 'gbdt',\n",
" 'objective': 'regression',\n",
" \"metric\": 'rmse'}\n",
" for key, value in trial.params.items():\n",
" params[key]=value\n",
" \n",
" dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
" dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
" model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
" verbose_eval=50,\n",
" early_stopping_rounds=20,\n",
" num_boost_round=5000)\n",
" test_predict[:,i] = model.predict(test_x)\n",
" predict = np.mean(test_predict,axis=1)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "f28d82da",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.432341765333029"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 评估指标rmse，越小越好\n",
"np.sqrt(mean_squared_error(test_y,predict))"
]
},
{
"cell_type": "markdown",
"id": "0c7ee3c5",
"metadata": {},
"source": [
"### 回归任务的结论\n",
"不使用optuna的RMSE分数是0.4346521330333544，使用的RMSE分数是0.432341765333029，提升了0.0023103677003254。\n",
"\n",
"作者测了很多次，基本在0.003-0.002之间，感兴趣的可以多跑几次。"
]
},
{
"cell_type": "markdown",
"id": "bb67b354",
"metadata": {},
"source": [
"## 森林植被类型（多分类）"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "108949a0",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import time\n",
"import gc\n",
"\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.metrics import accuracy_score\n",
"from sklearn.preprocessing import OrdinalEncoder\n",
"import lightgbm as lgb # 使用lgb模型"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "36e26941",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"七分类任务，处理前： [1 2 3 4 5 6 7]\n",
"[5 5 2 ... 3 3 3]\n",
"七分类任务，处理后： [0. 1. 2. 3. 4. 5. 6.]\n",
"[4. 4. 1. ... 2. 2. 2.]\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Elevation</th>\n",
" <th>Aspect</th>\n",
" <th>Slope</th>\n",
" <th>Horizontal_Distance_To_Hydrology</th>\n",
" <th>Vertical_Distance_To_Hydrology</th>\n",
" <th>Horizontal_Distance_To_Roadways</th>\n",
" <th>Hillshade_9am</th>\n",
" <th>Hillshade_Noon</th>\n",
" <th>Hillshade_3pm</th>\n",
" <th>Horizontal_Distance_To_Fire_Points</th>\n",
" <th>...</th>\n",
" <th>Soil_Type_30</th>\n",
" <th>Soil_Type_31</th>\n",
" <th>Soil_Type_32</th>\n",
" <th>Soil_Type_33</th>\n",
" <th>Soil_Type_34</th>\n",
" <th>Soil_Type_35</th>\n",
" <th>Soil_Type_36</th>\n",
" <th>Soil_Type_37</th>\n",
" <th>Soil_Type_38</th>\n",
" <th>Soil_Type_39</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>2596.0</td>\n",
" <td>51.0</td>\n",
" <td>3.0</td>\n",
" <td>258.0</td>\n",
" <td>0.0</td>\n",
" <td>510.0</td>\n",
" <td>221.0</td>\n",
" <td>232.0</td>\n",
" <td>148.0</td>\n",
" <td>6279.0</td>\n",
" <td>...</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2590.0</td>\n",
" <td>56.0</td>\n",
" <td>2.0</td>\n",
" <td>212.0</td>\n",
" <td>-6.0</td>\n",
" <td>390.0</td>\n",
" <td>220.0</td>\n",
" <td>235.0</td>\n",
" <td>151.0</td>\n",
" <td>6225.0</td>\n",
" <td>...</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>2 rows × 54 columns</p>\n",
"</div>"
],
"text/plain": [
" Elevation Aspect Slope Horizontal_Distance_To_Hydrology \\\n",
"0 2596.0 51.0 3.0 258.0 \n",
"1 2590.0 56.0 2.0 212.0 \n",
"\n",
" Vertical_Distance_To_Hydrology Horizontal_Distance_To_Roadways \\\n",
"0 0.0 510.0 \n",
"1 -6.0 390.0 \n",
"\n",
" Hillshade_9am Hillshade_Noon Hillshade_3pm \\\n",
"0 221.0 232.0 148.0 \n",
"1 220.0 235.0 151.0 \n",
"\n",
" Horizontal_Distance_To_Fire_Points ... Soil_Type_30 Soil_Type_31 \\\n",
"0 6279.0 ... 0.0 0.0 \n",
"1 6225.0 ... 0.0 0.0 \n",
"\n",
" Soil_Type_32 Soil_Type_33 Soil_Type_34 Soil_Type_35 Soil_Type_36 \\\n",
"0 0.0 0.0 0.0 0.0 0.0 \n",
"1 0.0 0.0 0.0 0.0 0.0 \n",
"\n",
" Soil_Type_37 Soil_Type_38 Soil_Type_39 \n",
"0 0.0 0.0 0.0 \n",
"1 0.0 0.0 0.0 \n",
"\n",
"[2 rows x 54 columns]"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.datasets import fetch_covtype\n",
"data = fetch_covtype()\n",
"X, y = data['data'], data['target']\n",
"X = pd.DataFrame(X,columns=data.feature_names)\n",
"print('七分类任务，处理前：',np.unique(y))\n",
"print(y)\n",
"ord = OrdinalEncoder()\n",
"y = ord.fit_transform(y.reshape(-1, 1))\n",
"y = y.reshape(-1, )\n",
"print('七分类任务，处理后：',np.unique(y))\n",
"print(y)\n",
"X.head(2)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "152a8658",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"RangeIndex: 581012 entries, 0 to 581011\n",
"Data columns (total 54 columns):\n",
" # Column Non-Null Count Dtype \n",
"--- ------ -------------- ----- \n",
" 0 Elevation 581012 non-null float64\n",
" 1 Aspect 581012 non-null float64\n",
" 2 Slope 581012 non-null float64\n",
" 3 Horizontal_Distance_To_Hydrology 581012 non-null float64\n",
" 4 Vertical_Distance_To_Hydrology 581012 non-null float64\n",
" 5 Horizontal_Distance_To_Roadways 581012 non-null float64\n",
" 6 Hillshade_9am 581012 non-null float64\n",
" 7 Hillshade_Noon 581012 non-null float64\n",
" 8 Hillshade_3pm 581012 non-null float64\n",
" 9 Horizontal_Distance_To_Fire_Points 581012 non-null float64\n",
" 10 Wilderness_Area_0 581012 non-null float64\n",
" 11 Wilderness_Area_1 581012 non-null float64\n",
" 12 Wilderness_Area_2 581012 non-null float64\n",
" 13 Wilderness_Area_3 581012 non-null float64\n",
" 14 Soil_Type_0 581012 non-null float64\n",
" 15 Soil_Type_1 581012 non-null float64\n",
" 16 Soil_Type_2 581012 non-null float64\n",
" 17 Soil_Type_3 581012 non-null float64\n",
" 18 Soil_Type_4 581012 non-null float64\n",
" 19 Soil_Type_5 581012 non-null float64\n",
" 20 Soil_Type_6 581012 non-null float64\n",
" 21 Soil_Type_7 581012 non-null float64\n",
" 22 Soil_Type_8 581012 non-null float64\n",
" 23 Soil_Type_9 581012 non-null float64\n",
" 24 Soil_Type_10 581012 non-null float64\n",
" 25 Soil_Type_11 581012 non-null float64\n",
" 26 Soil_Type_12 581012 non-null float64\n",
" 27 Soil_Type_13 581012 non-null float64\n",
" 28 Soil_Type_14 581012 non-null float64\n",
" 29 Soil_Type_15 581012 non-null float64\n",
" 30 Soil_Type_16 581012 non-null float64\n",
" 31 Soil_Type_17 581012 non-null float64\n",
" 32 Soil_Type_18 581012 non-null float64\n",
" 33 Soil_Type_19 581012 non-null float64\n",
" 34 Soil_Type_20 581012 non-null float64\n",
" 35 Soil_Type_21 581012 non-null float64\n",
" 36 Soil_Type_22 581012 non-null float64\n",
" 37 Soil_Type_23 581012 non-null float64\n",
" 38 Soil_Type_24 581012 non-null float64\n",
" 39 Soil_Type_25 581012 non-null float64\n",
" 40 Soil_Type_26 581012 non-null float64\n",
" 41 Soil_Type_27 581012 non-null float64\n",
" 42 Soil_Type_28 581012 non-null float64\n",
" 43 Soil_Type_29 581012 non-null float64\n",
" 44 Soil_Type_30 581012 non-null float64\n",
" 45 Soil_Type_31 581012 non-null float64\n",
" 46 Soil_Type_32 581012 non-null float64\n",
" 47 Soil_Type_33 581012 non-null float64\n",
" 48 Soil_Type_34 581012 non-null float64\n",
" 49 Soil_Type_35 581012 non-null float64\n",
" 50 Soil_Type_36 581012 non-null float64\n",
" 51 Soil_Type_37 581012 non-null float64\n",
" 52 Soil_Type_38 581012 non-null float64\n",
" 53 Soil_Type_39 581012 non-null float64\n",
"dtypes: float64(54)\n",
"memory usage: 239.4 MB\n"
]
}
],
"source": [
"X.info()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "67086c1c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"训练集： (522910, 54)\n",
"测试集： (58102, 54)\n"
]
}
],
"source": [
"# 切分训练和测试集\n",
"train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n",
"train_y = pd.DataFrame(train_y,columns=['result'])\n",
"test_y = pd.DataFrame(test_y,columns=['result'])\n",
"\n",
"print('训练集：',train_x.shape)\n",
"print('测试集：', test_x.shape)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "88350615",
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import StratifiedKFold\n",
"skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)"
]
},
{
"cell_type": "code",
"execution_count": 58,
"id": "2e694b0c",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.030610 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2264\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.347629\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508300\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.424906\n",
"[100]\tvalid_0's multi_logloss: 0.413801\n",
"Early stopping, best iteration is:\n",
"[90]\tvalid_0's multi_logloss: 0.37472\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031196 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2269\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.347629\n",
"[LightGBM] [Info] Start training from score -4.112309\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.442955\n",
"[100]\tvalid_0's multi_logloss: 0.396755\n",
"[150]\tvalid_0's multi_logloss: 0.378273\n",
"Early stopping, best iteration is:\n",
"[135]\tvalid_0's multi_logloss: 0.375671\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.018390 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2260\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.431872\n",
"[100]\tvalid_0's multi_logloss: 0.378335\n",
"Early stopping, best iteration is:\n",
"[109]\tvalid_0's multi_logloss: 0.37075\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.032231 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2262\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009528\n",
"[LightGBM] [Info] Start training from score -0.717945\n",
"[LightGBM] [Info] Start training from score -2.789114\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343871\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.437033\n",
"[100]\tvalid_0's multi_logloss: 0.387111\n",
"Early stopping, best iteration is:\n",
"[109]\tvalid_0's multi_logloss: 0.380009\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031067 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2268\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009528\n",
"[LightGBM] [Info] Start training from score -0.717945\n",
"[LightGBM] [Info] Start training from score -2.789114\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343871\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.428647\n",
"[100]\tvalid_0's multi_logloss: 0.485091\n",
"Early stopping, best iteration is:\n",
"[85]\tvalid_0's multi_logloss: 0.386918\n",
"Wall time: 1min 53s\n"
]
}
],
"source": [
"%%time\n",
"test_predict = 0\n",
"params = {\"boosting_type\": \"gbdt\",\n",
" \"objective\": \"multiclass\",\n",
" \"num_class': 7,\n",
" \"metric\": \"multi_logloss\"}\n",
"for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
" dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
" dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
" model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
" verbose_eval=50,\n",
" early_stopping_rounds=20,\n",
" num_boost_round=5000)\n",
" test_predict += model.predict(test_x)\n",
"predict = test_predict/5"
]
},
{
"cell_type": "code",
"execution_count": 62,
"id": "6d7d4447",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"不使用optuna的多分类分数： 0.8631716636260369\n"
]
}
],
"source": [
"# 评估指标acc，越大越好\n",
"print('不使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))"
]
},
{
"cell_type": "code",
"execution_count": 70,
"id": "f6571309",
"metadata": {},
"outputs": [],
"source": [
"import optuna\n",
"def objective(trial,train_x, train_y, valid_x, valid_y):\n",
" dtrain = lgb.Dataset(train_x, label=train_y)\n",
" dvalid = lgb.Dataset(valid_x, label=valid_y)\n",
"\n",
" param = {\n",
" \"objective\": \"multiclass\",\n",
" \"metric\": \"multi_logloss\",\n",
" \"verbosity\": -1,\n",
" \"num_class\": 7,\n",
" \"boosting_type\": \"gbdt\",\n",
" 'random_state':42,\n",
" \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n",
" \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n",
" \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n",
" \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n",
" \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n",
" \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n",
" \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n",
" }\n",
"\n",
" # Add a callback for pruning.\n",
" pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"multi_logloss\")\n",
" gbm = lgb.train(\n",
" param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n",
" )\n",
"\n",
" preds = gbm.predict(valid_x)\n",
" pred_labels = np.rint(preds)\n",
" acc = accuracy_score(np.argmax(pred_labels, axis=1), valid_y)\n",
" return acc"
]
},
{
"cell_type": "code",
"execution_count": 72,
"id": "4170d934",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 18:21:48,373]\u001b[0m A new study created in memory with name: no-name-7584aee7-d7a5-42ce-8426-7d4c7e034372\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:22:17,890]\u001b[0m Trial 0 finished with value: 0.8652253733912145 and parameters: {'lambda_l1': 0.00029350441808598135, 'lambda_l2': 0.0050140304157587625, 'num_leaves': 41, 'feature_fraction': 0.7195254545980898, 'bagging_fraction': 0.6783357374559799, 'bagging_freq': 6, 'min_child_samples': 32}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:22:40,277]\u001b[0m Trial 1 finished with value: 0.865770400260083 and parameters: {'lambda_l1': 8.073501039689362e-07, 'lambda_l2': 2.7820830665481296e-05, 'num_leaves': 41, 'feature_fraction': 0.8160512302334224, 'bagging_fraction': 0.6156688842906214, 'bagging_freq': 7, 'min_child_samples': 35}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:23:14,303]\u001b[0m Trial 2 finished with value: 0.9372358532060967 and parameters: {'lambda_l1': 1.1987047256838896e-07, 'lambda_l2': 0.0008429364945676234, 'num_leaves': 203, 'feature_fraction': 0.6245432353876832, 'bagging_fraction': 0.6680546579110309, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:23:37,365]\u001b[0m Trial 3 finished with value: 0.8423533686485246 and parameters: {'lambda_l1': 6.031700696477528e-05, 'lambda_l2': 0.24394450720600092, 'num_leaves': 33, 'feature_fraction': 0.48861458318031864, 'bagging_fraction': 0.9555014696451426, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:24:15,185]\u001b[0m Trial 4 finished with value: 0.9382876594442638 and parameters: {'lambda_l1': 0.013655390719191782, 'lambda_l2': 2.248487596614776e-07, 'num_leaves': 229, 'feature_fraction': 0.6314327078814898, 'bagging_fraction': 0.9426524187528247, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:24:18,728]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:25:04,830]\u001b[0m Trial 6 finished with value: 0.9459084737335297 and parameters: {'lambda_l1': 0.02773561511163358, 'lambda_l2': 0.30726353465031164, 'num_leaves': 238, 'feature_fraction': 0.8197796474237233, 'bagging_fraction': 0.9951387365343576, 'bagging_freq': 7, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:25:10,692]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:25:52,087]\u001b[0m Trial 8 finished with value: 0.9404582050448452 and parameters: {'lambda_l1': 2.969808205332883e-07, 'lambda_l2': 3.0108163042597293e-05, 'num_leaves': 184, 'feature_fraction': 0.9858004420758661, 'bagging_fraction': 0.7201379680285598, 'bagging_freq': 2, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:25:57,802]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:02,460]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:07,268]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:11,252]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:15,570]\u001b[0m Trial 13 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:21,086]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:23,907]\u001b[0m Trial 15 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:27,922]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:33,712]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:37,313]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:42,273]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:46,859]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:50,342]\u001b[0m Trial 21 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:55,710]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:26:59,948]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:03,848]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:08,309]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:13,458]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:16,591]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:21,016]\u001b[0m Trial 28 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:24,997]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:32,574]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 19.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:37,797]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:27:43,562]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:28:21,369]\u001b[0m Trial 33 finished with value: 0.9396167600543115 and parameters: {'lambda_l1': 1.7053800672111565e-08, 'lambda_l2': 0.009838404261194042, 'num_leaves': 205, 'feature_fraction': 0.8042641806396126, 'bagging_fraction': 0.4068986434462453, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:28:28,805]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:28:32,440]\u001b[0m Trial 35 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:29:16,242]\u001b[0m Trial 36 finished with value: 0.9376948232009332 and parameters: {'lambda_l1': 0.00025741391284768883, 'lambda_l2': 1.3903821165130423e-05, 'num_leaves': 179, 'feature_fraction': 0.7513521003294499, 'bagging_fraction': 0.6877869004228008, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:29:22,290]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:29:27,666]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:29:33,842]\u001b[0m Trial 39 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:29:38,984]\u001b[0m Trial 40 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:30:13,915]\u001b[0m Trial 41 finished with value: 0.9370828632078178 and parameters: {'lambda_l1': 0.0003772109478239288, 'lambda_l2': 1.4609599162418084e-05, 'num_leaves': 177, 'feature_fraction': 0.7423326328000164, 'bagging_fraction': 0.6856961600540766, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:30:48,039]\u001b[0m Trial 42 finished with value: 0.9382972213191563 and parameters: {'lambda_l1': 0.0005140907001059531, 'lambda_l2': 2.4931963200658e-05, 'num_leaves': 191, 'feature_fraction': 0.7220599193293482, 'bagging_fraction': 0.6449224331658816, 'bagging_freq': 1, 'min_child_samples': 65}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:30:52,499]\u001b[0m Trial 43 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:31:27,381]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 93.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:31:32,050]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:04,026]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 94.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:08,636]\u001b[0m Trial 47 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:14,065]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:17,136]\u001b[0m Trial 49 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:22,164]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:32:56,694]\u001b[0m Trial 51 finished with value: 0.9398653688015146 and parameters: {'lambda_l1': 8.007598044441421e-05, 'lambda_l2': 1.9006501098879968e-05, 'num_leaves': 193, 'feature_fraction': 0.7474479951356153, 'bagging_fraction': 0.6831044198923434, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 18:33:02,134]\u001b[0m Trial 52 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:25,476]\u001b[0m Trial 53 pruned. Trial was pruned at iteration 75.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:30,367]\u001b[0m Trial 54 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:34,302]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:40,033]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:43,381]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:47,728]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:52,240]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:33:56,752]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:34:02,449]\u001b[0m Trial 61 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:34:07,140]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:34:13,623]\u001b[0m Trial 63 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:34:19,092]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:34:57,096]\u001b[0m Trial 65 finished with value: 0.945717236235681 and parameters: {'lambda_l1': 0.0006215692281330095, 'lambda_l2': 0.0025507405649904265, 'num_leaves': 254, 'feature_fraction': 0.684182512525035, 'bagging_fraction': 0.881844626841578, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:04,313]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:08,810]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:13,882]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:18,314]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:21,402]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:35:26,827]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:06,776]\u001b[0m Trial 72 finished with value: 0.9417586200302155 and parameters: {'lambda_l1': 0.0008708841660641308, 'lambda_l2': 5.4043063920808084e-05, 'num_leaves': 224, 'feature_fraction': 0.7182774647861223, 'bagging_fraction': 0.6574176618253137, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:10,840]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:16,482]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:45,510]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 85.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:50,598]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:36:56,577]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:01,576]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:05,793]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:41,797]\u001b[0m Trial 80 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.08921307105729769, 'lambda_l2': 4.975883016961999e-06, 'num_leaves': 214, 'feature_fraction': 0.8281286566426087, 'bagging_fraction': 0.5953173368434432, 'bagging_freq': 1, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:46,363]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:51,749]\u001b[0m Trial 82 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:37:56,140]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:38:00,762]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:38:06,090]\u001b[0m Trial 85 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:38:10,580]\u001b[0m Trial 86 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:38:48,850]\u001b[0m Trial 87 finished with value: 0.9479260293358321 and parameters: {'lambda_l1': 0.02890836192523439, 'lambda_l2': 4.025111402953753e-05, 'num_leaves': 247, 'feature_fraction': 0.8727646045462183, 'bagging_fraction': 0.7437461253889492, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:38:54,311]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:39:36,775]\u001b[0m Trial 89 finished with value: 0.9436136237593468 and parameters: {'lambda_l1': 2.776285425246586e-05, 'lambda_l2': 3.1180901688950096, 'num_leaves': 235, 'feature_fraction': 0.8371612462873237, 'bagging_fraction': 0.6753094794878766, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:39:41,160]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:15,410]\u001b[0m Trial 91 finished with value: 0.9393968369317856 and parameters: {'lambda_l1': 7.819588207346975e-05, 'lambda_l2': 9.678552881824005e-06, 'num_leaves': 191, 'feature_fraction': 0.7476569605668175, 'bagging_fraction': 0.6665662519078225, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:20,661]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:23,908]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:30,385]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:34,950]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:41,586]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:46,857]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:49,830]\u001b[0m Trial 98 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:40:55,460]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 100\n",
"Best trial:\n",
" Value: 0.8423533686485246\n",
" Params: \n",
" lambda_l1: 6.031700696477528e-05\n",
" lambda_l2: 0.24394450720600092\n",
" num_leaves: 33\n",
" feature_fraction: 0.48861458318031864\n",
" bagging_fraction: 0.9555014696451426\n",
" bagging_freq: 1\n",
" min_child_samples: 79\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.023061 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2262\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.347629\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508300\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.444681\n",
"[100]\tvalid_0's multi_logloss: 0.37701\n",
"[150]\tvalid_0's multi_logloss: 0.33841\n",
"[200]\tvalid_0's multi_logloss: 0.311971\n",
"[250]\tvalid_0's multi_logloss: 0.291695\n",
"[300]\tvalid_0's multi_logloss: 0.274889\n",
"[350]\tvalid_0's multi_logloss: 0.261281\n",
"[400]\tvalid_0's multi_logloss: 0.250771\n",
"[450]\tvalid_0's multi_logloss: 0.240104\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[500]\tvalid_0's multi_logloss: 0.230589\n",
"[550]\tvalid_0's multi_logloss: 0.222278\n",
"[600]\tvalid_0's multi_logloss: 0.215255\n",
"[650]\tvalid_0's multi_logloss: 0.208483\n",
"[700]\tvalid_0's multi_logloss: 0.202692\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[750]\tvalid_0's multi_logloss: 0.196167\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[800]\tvalid_0's multi_logloss: 0.190411\n",
"[850]\tvalid_0's multi_logloss: 0.185117\n",
"[900]\tvalid_0's multi_logloss: 0.18105\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[950]\tvalid_0's multi_logloss: 0.176702\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1000]\tvalid_0's multi_logloss: 0.172928\n",
"[1050]\tvalid_0's multi_logloss: 0.169457\n",
"[1100]\tvalid_0's multi_logloss: 0.165919\n",
"[1150]\tvalid_0's multi_logloss: 0.163006\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1200]\tvalid_0's multi_logloss: 0.15967\n",
"[1250]\tvalid_0's multi_logloss: 0.156712\n",
"[1300]\tvalid_0's multi_logloss: 0.154223\n",
"[1350]\tvalid_0's multi_logloss: 0.151783\n",
"[1400]\tvalid_0's multi_logloss: 0.149313\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1450]\tvalid_0's multi_logloss: 0.147362\n",
"[1500]\tvalid_0's multi_logloss: 0.145167\n",
"[1550]\tvalid_0's multi_logloss: 0.143229\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1600]\tvalid_0's multi_logloss: 0.141306\n",
"[1650]\tvalid_0's multi_logloss: 0.139419\n",
"[1700]\tvalid_0's multi_logloss: 0.137686\n",
"[1750]\tvalid_0's multi_logloss: 0.136191\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1800]\tvalid_0's multi_logloss: 0.134525\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1850]\tvalid_0's multi_logloss: 0.13314\n",
"[1900]\tvalid_0's multi_logloss: 0.131585\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1950]\tvalid_0's multi_logloss: 0.130162\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2000]\tvalid_0's multi_logloss: 0.128702\n",
"[2050]\tvalid_0's multi_logloss: 0.127237\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2100]\tvalid_0's multi_logloss: 0.125969\n",
"[2150]\tvalid_0's multi_logloss: 0.125018\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2200]\tvalid_0's multi_logloss: 0.124019\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2250]\tvalid_0's multi_logloss: 0.122787\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2300]\tvalid_0's multi_logloss: 0.121904\n",
"[2350]\tvalid_0's multi_logloss: 0.120813\n",
"[2400]\tvalid_0's multi_logloss: 0.119856\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2450]\tvalid_0's multi_logloss: 0.119105\n",
"[2500]\tvalid_0's multi_logloss: 0.118181\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2550]\tvalid_0's multi_logloss: 0.117233\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2600]\tvalid_0's multi_logloss: 0.11653\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2650]\tvalid_0's multi_logloss: 0.115597\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2700]\tvalid_0's multi_logloss: 0.114851\n",
"[2750]\tvalid_0's multi_logloss: 0.114109\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2800]\tvalid_0's multi_logloss: 0.113475\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2850]\tvalid_0's multi_logloss: 0.112871\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2900]\tvalid_0's multi_logloss: 0.112261\n",
"[2950]\tvalid_0's multi_logloss: 0.111637\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3000]\tvalid_0's multi_logloss: 0.111155\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3050]\tvalid_0's multi_logloss: 0.110524\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3100]\tvalid_0's multi_logloss: 0.109987\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3150]\tvalid_0's multi_logloss: 0.1095\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3200]\tvalid_0's multi_logloss: 0.108922\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3250]\tvalid_0's multi_logloss: 0.108389\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3300]\tvalid_0's multi_logloss: 0.107919\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[3350]\tvalid_0's multi_logloss: 0.107463\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3400]\tvalid_0's multi_logloss: 0.107071\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3450]\tvalid_0's multi_logloss: 0.106658\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3500]\tvalid_0's multi_logloss: 0.106334\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3550]\tvalid_0's multi_logloss: 0.105908\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3600]\tvalid_0's multi_logloss: 0.105497\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3650]\tvalid_0's multi_logloss: 0.105035\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3700]\tvalid_0's multi_logloss: 0.104725\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3750]\tvalid_0's multi_logloss: 0.104382\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3800]\tvalid_0's multi_logloss: 0.104083\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3850]\tvalid_0's multi_logloss: 0.103648\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3900]\tvalid_0's multi_logloss: 0.103237\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3950]\tvalid_0's multi_logloss: 0.102936\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4000]\tvalid_0's multi_logloss: 0.102547\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4050]\tvalid_0's multi_logloss: 0.102233\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4100]\tvalid_0's multi_logloss: 0.102018\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4150]\tvalid_0's multi_logloss: 0.10177\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4200]\tvalid_0's multi_logloss: 0.101442\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4250]\tvalid_0's multi_logloss: 0.101199\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4300]\tvalid_0's multi_logloss: 0.101029\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4350]\tvalid_0's multi_logloss: 0.100696\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4400]\tvalid_0's multi_logloss: 0.100538\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4450]\tvalid_0's multi_logloss: 0.100337\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4500]\tvalid_0's multi_logloss: 0.100081\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4550]\tvalid_0's multi_logloss: 0.0998246\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4600]\tvalid_0's multi_logloss: 0.0996299\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4650]\tvalid_0's multi_logloss: 0.0994822\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4700]\tvalid_0's multi_logloss: 0.0992758\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4750]\tvalid_0's multi_logloss: 0.0989493\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4800]\tvalid_0's multi_logloss: 0.098729\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4850]\tvalid_0's multi_logloss: 0.0984868\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4900]\tvalid_0's multi_logloss: 0.0983602\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4950]\tvalid_0's multi_logloss: 0.0982792\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[5000]\tvalid_0's multi_logloss: 0.0981563\n",
"Did not meet early stopping. Best iteration is:\n",
"[4998]\tvalid_0's multi_logloss: 0.098148\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 18:56:02,686]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:56:09,367]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:56:45,840]\u001b[0m Trial 102 finished with value: 0.9407450612916181 and parameters: {'lambda_l1': 8.197882698131674e-05, 'lambda_l2': 0.00021208640393700757, 'num_leaves': 192, 'feature_fraction': 0.7731692868777431, 'bagging_fraction': 0.6735311455374017, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:56:50,126]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:57:26,832]\u001b[0m Trial 104 finished with value: 0.9414526400336578 and parameters: {'lambda_l1': 1.2795362377766212e-05, 'lambda_l2': 1.8844145747184915e-05, 'num_leaves': 217, 'feature_fraction': 0.7191746432394106, 'bagging_fraction': 0.7098988463324459, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:57:32,559]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:57:37,952]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:57:40,971]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:57:45,702]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:58:22,795]\u001b[0m Trial 109 finished with value: 0.942236713774837 and parameters: {'lambda_l1': 2.209464573982675e-06, 'lambda_l2': 8.287334662719688e-06, 'num_leaves': 207, 'feature_fraction': 0.7665224568657326, 'bagging_fraction': 0.6416358656740727, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:58:27,752]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:58:32,799]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:01,076]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 88.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:06,071]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:11,302]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:15,597]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:19,903]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 18:59:33,007]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 37.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:00:13,094]\u001b[0m Trial 118 finished with value: 0.9443212025013865 and parameters: {'lambda_l1': 0.002942524852825639, 'lambda_l2': 0.011610635505416795, 'num_leaves': 203, 'feature_fraction': 0.9531605764605278, 'bagging_fraction': 0.630189454389572, 'bagging_freq': 4, 'min_child_samples': 42}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:00:17,760]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:00:23,215]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:00:57,349]\u001b[0m Trial 121 finished with value: 0.9405347000439847 and parameters: {'lambda_l1': 7.143416155152381e-05, 'lambda_l2': 0.00017466919367877497, 'num_leaves': 193, 'feature_fraction': 0.7734033889080848, 'bagging_fraction': 0.6713509524709522, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:01:33,315]\u001b[0m Trial 122 finished with value: 0.9424757606471477 and parameters: {'lambda_l1': 3.671159464706789e-05, 'lambda_l2': 3.9461792591530047e-05, 'num_leaves': 209, 'feature_fraction': 0.7582936549212366, 'bagging_fraction': 0.6671570260041155, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:01:39,469]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:15,641]\u001b[0m Trial 124 finished with value: 0.9422271518999445 and parameters: {'lambda_l1': 9.126230817825131e-08, 'lambda_l2': 9.42798578840402e-06, 'num_leaves': 212, 'feature_fraction': 0.7770330244830586, 'bagging_fraction': 0.6480274535778553, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:20,777]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:26,240]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:45,146]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 61.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:50,386]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:53,658]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:02:58,360]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:03:15,101]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 50.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:03:50,148]\u001b[0m Trial 132 finished with value: 0.9417873056548928 and parameters: {'lambda_l1': 8.072026862454635e-05, 'lambda_l2': 8.510873972830464e-05, 'num_leaves': 201, 'feature_fraction': 0.7644190714627204, 'bagging_fraction': 0.679994557223264, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:03:54,573]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:03:59,616]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:03,756]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:07,932]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:14,765]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:19,167]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:54,001]\u001b[0m Trial 139 finished with value: 0.942370580023331 and parameters: {'lambda_l1': 3.232550131286934e-06, 'lambda_l2': 0.0006916472301971968, 'num_leaves': 202, 'feature_fraction': 0.8343548844441779, 'bagging_fraction': 0.8476577552042586, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:04:58,456]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:05:04,372]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:05:41,758]\u001b[0m Trial 142 finished with value: 0.9466256143504619 and parameters: {'lambda_l1': 2.980896851114429e-05, 'lambda_l2': 2.913417349261647e-05, 'num_leaves': 239, 'feature_fraction': 0.763135869473294, 'bagging_fraction': 0.7513284296958156, 'bagging_freq': 1, 'min_child_samples': 62}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:05:46,991]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:05:53,817]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:06:30,154]\u001b[0m Trial 145 finished with value: 0.9437474900078408 and parameters: {'lambda_l1': 7.212460033828516e-05, 'lambda_l2': 4.309094661715439e-06, 'num_leaves': 212, 'feature_fraction': 0.7533850653260141, 'bagging_fraction': 0.8916021652632435, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:06,486]\u001b[0m Trial 146 finished with value: 0.9439291656307969 and parameters: {'lambda_l1': 0.0005892869464862208, 'lambda_l2': 1.1242672082140708e-05, 'num_leaves': 223, 'feature_fraction': 0.7312666460187671, 'bagging_fraction': 0.6736774370139369, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:10,387]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:14,676]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:20,110]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:24,141]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:07:28,653]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:08:04,981]\u001b[0m Trial 152 finished with value: 0.9427626168939206 and parameters: {'lambda_l1': 0.0011892669983091146, 'lambda_l2': 5.820028108882431e-05, 'num_leaves': 224, 'feature_fraction': 0.7351610234838635, 'bagging_fraction': 0.6511917047811185, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 19:08:41,831]\u001b[0m Trial 153 finished with value: 0.9482989424566369 and parameters: {'lambda_l1': 0.00037447863194847096, 'lambda_l2': 2.1542282896260357e-05, 'num_leaves': 234, 'feature_fraction': 0.9144698353599378, 'bagging_fraction': 0.6832488195447302, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:08:47,021]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:08:52,838]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:08:58,447]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:09:02,729]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:09:09,544]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:09:13,126]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:09:40,786]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 88.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:09:56,557]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 42.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:01,280]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:37,093]\u001b[0m Trial 163 finished with value: 0.9425331318965022 and parameters: {'lambda_l1': 6.397677155805411e-08, 'lambda_l2': 0.00039621691928490413, 'num_leaves': 206, 'feature_fraction': 0.7667580883039189, 'bagging_fraction': 0.657903822017336, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:41,857]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:47,583]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:52,424]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:10:58,913]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:11:03,909]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:11:09,381]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:11:13,686]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:11:49,897]\u001b[0m Trial 171 finished with value: 0.9430112256411237 and parameters: {'lambda_l1': 1.102134869211263e-07, 'lambda_l2': 7.503180286760575e-06, 'num_leaves': 214, 'feature_fraction': 0.783389889391717, 'bagging_fraction': 0.6498041172731315, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:12:26,420]\u001b[0m Trial 172 finished with value: 0.9431355300147253 and parameters: {'lambda_l1': 1.0270752816234116e-07, 'lambda_l2': 1.0817935884223942e-05, 'num_leaves': 211, 'feature_fraction': 0.7583412868303638, 'bagging_fraction': 0.6795874395289168, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:13:04,383]\u001b[0m Trial 173 finished with value: 0.9439004800061196 and parameters: {'lambda_l1': 8.279755593895644e-08, 'lambda_l2': 3.6313073067998552e-06, 'num_leaves': 230, 'feature_fraction': 0.7727565832075506, 'bagging_fraction': 0.6452464868388876, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:13:10,975]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:13:46,828]\u001b[0m Trial 175 finished with value: 0.9445315637490199 and parameters: {'lambda_l1': 0.0002353217979236286, 'lambda_l2': 3.171263348326624e-05, 'num_leaves': 218, 'feature_fraction': 0.8203542425794451, 'bagging_fraction': 0.6588367468604717, 'bagging_freq': 1, 'min_child_samples': 76}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:13:51,827]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:13:55,900]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:14:32,835]\u001b[0m Trial 178 finished with value: 0.9489778355739994 and parameters: {'lambda_l1': 1.0560192692698665e-08, 'lambda_l2': 0.0002765136930724561, 'num_leaves': 246, 'feature_fraction': 0.7669065013864841, 'bagging_fraction': 0.6258778389490884, 'bagging_freq': 1, 'min_child_samples': 25}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:15:08,531]\u001b[0m Trial 179 finished with value: 0.9418638006540322 and parameters: {'lambda_l1': 5.8099076285804865e-08, 'lambda_l2': 9.604379213115968e-06, 'num_leaves': 208, 'feature_fraction': 0.7900315064460727, 'bagging_fraction': 0.6778057274702293, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:15:14,384]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:15:22,172]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:15:32,935]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 32.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:16:09,350]\u001b[0m Trial 183 finished with value: 0.9439865368801514 and parameters: {'lambda_l1': 7.039591924287473e-08, 'lambda_l2': 2.5146082118354423e-05, 'num_leaves': 221, 'feature_fraction': 0.7397390119318659, 'bagging_fraction': 0.672737210757752, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:16:13,980]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:16:18,553]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:16:55,502]\u001b[0m Trial 186 finished with value: 0.9441682125031076 and parameters: {'lambda_l1': 1.9294698699801913e-07, 'lambda_l2': 2.5570868408037523e-06, 'num_leaves': 216, 'feature_fraction': 0.7829248911740091, 'bagging_fraction': 0.9993734316792878, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:01,011]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:04,368]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:09,352]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:15,404]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:20,157]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:17:56,849]\u001b[0m Trial 192 finished with value: 0.9449427243693943 and parameters: {'lambda_l1': 3.0670346509523553e-06, 'lambda_l2': 7.037461956773899e-06, 'num_leaves': 226, 'feature_fraction': 0.7927923736502508, 'bagging_fraction': 0.6369345538445389, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:18:02,512]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:18:06,965]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:18:42,532]\u001b[0m Trial 195 finished with value: 0.941882924403817 and parameters: {'lambda_l1': 5.448535054846655e-05, 'lambda_l2': 0.008712731969886423, 'num_leaves': 211, 'feature_fraction': 0.7730826357400354, 'bagging_fraction': 0.6634899720026096, 'bagging_freq': 1, 'min_child_samples': 90}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:19:18,834]\u001b[0m Trial 196 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 6.64569708412817e-05, 'lambda_l2': 0.0027368345553249167, 'num_leaves': 212, 'feature_fraction': 0.8161286103449696, 'bagging_fraction': 0.6899947578536335, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:19:24,121]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:19:28,683]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:19:33,772]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 200\n",
"Best trial:\n",
" Value: 0.8423533686485246\n",
" Params: \n",
" lambda_l1: 6.031700696477528e-05\n",
" lambda_l2: 0.24394450720600092\n",
" num_leaves: 33\n",
" feature_fraction: 0.48861458318031864\n",
" bagging_fraction: 0.9555014696451426\n",
" bagging_freq: 1\n",
" min_child_samples: 79\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027211 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2267\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.347629\n",
"[LightGBM] [Info] Start training from score -4.112309\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.442887\n",
"[100]\tvalid_0's multi_logloss: 0.373911\n",
"[150]\tvalid_0's multi_logloss: 0.336668\n",
"[200]\tvalid_0's multi_logloss: 0.312308\n",
"[250]\tvalid_0's multi_logloss: 0.293877\n",
"[300]\tvalid_0's multi_logloss: 0.277526\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[350]\tvalid_0's multi_logloss: 0.262446\n",
"[400]\tvalid_0's multi_logloss: 0.251811\n",
"[450]\tvalid_0's multi_logloss: 0.240068\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[500]\tvalid_0's multi_logloss: 0.230934\n",
"[550]\tvalid_0's multi_logloss: 0.222786\n",
"[600]\tvalid_0's multi_logloss: 0.214897\n",
"[650]\tvalid_0's multi_logloss: 0.207044\n",
"[700]\tvalid_0's multi_logloss: 0.200602\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[750]\tvalid_0's multi_logloss: 0.194382\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[800]\tvalid_0's multi_logloss: 0.189488\n",
"[850]\tvalid_0's multi_logloss: 0.185057\n",
"[900]\tvalid_0's multi_logloss: 0.180852\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[950]\tvalid_0's multi_logloss: 0.17619\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1000]\tvalid_0's multi_logloss: 0.172508\n",
"[1050]\tvalid_0's multi_logloss: 0.169138\n",
"[1100]\tvalid_0's multi_logloss: 0.165975\n",
"[1150]\tvalid_0's multi_logloss: 0.162635\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1200]\tvalid_0's multi_logloss: 0.15997\n",
"[1250]\tvalid_0's multi_logloss: 0.156712\n",
"[1300]\tvalid_0's multi_logloss: 0.1538\n",
"[1350]\tvalid_0's multi_logloss: 0.151426\n",
"[1400]\tvalid_0's multi_logloss: 0.149565\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1450]\tvalid_0's multi_logloss: 0.147572\n",
"[1500]\tvalid_0's multi_logloss: 0.14554\n",
"[1550]\tvalid_0's multi_logloss: 0.143424\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1600]\tvalid_0's multi_logloss: 0.141495\n",
"[1650]\tvalid_0's multi_logloss: 0.139469\n",
"[1700]\tvalid_0's multi_logloss: 0.137832\n",
"[1750]\tvalid_0's multi_logloss: 0.13608\n",
"[1800]\tvalid_0's multi_logloss: 0.134508\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1850]\tvalid_0's multi_logloss: 0.132867\n",
"[1900]\tvalid_0's multi_logloss: 0.131194\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1950]\tvalid_0's multi_logloss: 0.1297\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2000]\tvalid_0's multi_logloss: 0.128321\n",
"[2050]\tvalid_0's multi_logloss: 0.127052\n",
"[2100]\tvalid_0's multi_logloss: 0.125839\n",
"[2150]\tvalid_0's multi_logloss: 0.124789\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2200]\tvalid_0's multi_logloss: 0.123598\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2250]\tvalid_0's multi_logloss: 0.122598\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2300]\tvalid_0's multi_logloss: 0.121569\n",
"[2350]\tvalid_0's multi_logloss: 0.120579\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2400]\tvalid_0's multi_logloss: 0.119715\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2450]\tvalid_0's multi_logloss: 0.118919\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2500]\tvalid_0's multi_logloss: 0.118045\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2550]\tvalid_0's multi_logloss: 0.117234\n",
"[2600]\tvalid_0's multi_logloss: 0.116567\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2650]\tvalid_0's multi_logloss: 0.115834\n",
"[2700]\tvalid_0's multi_logloss: 0.115171\n",
"[2750]\tvalid_0's multi_logloss: 0.114451\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2800]\tvalid_0's multi_logloss: 0.11376\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2850]\tvalid_0's multi_logloss: 0.113166\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2900]\tvalid_0's multi_logloss: 0.112467\n",
"[2950]\tvalid_0's multi_logloss: 0.111564\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3000]\tvalid_0's multi_logloss: 0.111056\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3050]\tvalid_0's multi_logloss: 0.110477\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3100]\tvalid_0's multi_logloss: 0.109703\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3150]\tvalid_0's multi_logloss: 0.109081\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3200]\tvalid_0's multi_logloss: 0.108637\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3250]\tvalid_0's multi_logloss: 0.108172\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3300]\tvalid_0's multi_logloss: 0.107737\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3350]\tvalid_0's multi_logloss: 0.107314\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3400]\tvalid_0's multi_logloss: 0.106692\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3450]\tvalid_0's multi_logloss: 0.106282\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3500]\tvalid_0's multi_logloss: 0.105917\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3550]\tvalid_0's multi_logloss: 0.105631\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3600]\tvalid_0's multi_logloss: 0.105321\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3650]\tvalid_0's multi_logloss: 0.104916\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3700]\tvalid_0's multi_logloss: 0.104489\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3750]\tvalid_0's multi_logloss: 0.104121\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3800]\tvalid_0's multi_logloss: 0.103784\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3850]\tvalid_0's multi_logloss: 0.103477\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3900]\tvalid_0's multi_logloss: 0.103191\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3950]\tvalid_0's multi_logloss: 0.102865\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4000]\tvalid_0's multi_logloss: 0.102531\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4050]\tvalid_0's multi_logloss: 0.102279\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4100]\tvalid_0's multi_logloss: 0.102028\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4150]\tvalid_0's multi_logloss: 0.101774\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4200]\tvalid_0's multi_logloss: 0.101443\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4250]\tvalid_0's multi_logloss: 0.101177\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4300]\tvalid_0's multi_logloss: 0.101004\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4350]\tvalid_0's multi_logloss: 0.100807\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4400]\tvalid_0's multi_logloss: 0.100584\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4450]\tvalid_0's multi_logloss: 0.100384\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4500]\tvalid_0's multi_logloss: 0.100211\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4550]\tvalid_0's multi_logloss: 0.0999913\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4600]\tvalid_0's multi_logloss: 0.0998812\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4650]\tvalid_0's multi_logloss: 0.0997132\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4700]\tvalid_0's multi_logloss: 0.0994868\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4750]\tvalid_0's multi_logloss: 0.0992342\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4800]\tvalid_0's multi_logloss: 0.0989926\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4850]\tvalid_0's multi_logloss: 0.0987583\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"Early stopping, best iteration is:\n",
"[4860]\tvalid_0's multi_logloss: 0.0987031\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 19:35:44,152]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:35:51,579]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:35:57,380]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:03,314]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:07,983]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:13,613]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:21,694]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:25,647]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:30,111]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:37,233]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:42,081]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:49,029]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:36:53,527]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:00,135]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:04,820]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:09,463]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:13,289]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:16,788]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:23,053]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:37:59,676]\u001b[0m Trial 219 finished with value: 0.9434319481363905 and parameters: {'lambda_l1': 1.1871087780899598e-07, 'lambda_l2': 0.00023968457905386466, 'num_leaves': 219, 'feature_fraction': 0.838039247899186, 'bagging_fraction': 0.6772774131242125, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:38:04,526]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:38:41,652]\u001b[0m Trial 221 finished with value: 0.9425618175211795 and parameters: {'lambda_l1': 8.509891041969528e-05, 'lambda_l2': 3.581606389921117e-05, 'num_leaves': 209, 'feature_fraction': 0.8843923083599923, 'bagging_fraction': 0.6707209079942898, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:38:50,823]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:38:55,262]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:00,904]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:06,074]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:10,767]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:15,341]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:19,652]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:39:57,821]\u001b[0m Trial 229 finished with value: 0.9456311793616492 and parameters: {'lambda_l1': 0.000357294778794299, 'lambda_l2': 4.383896339017896e-05, 'num_leaves': 218, 'feature_fraction': 0.9990787264144749, 'bagging_fraction': 0.6859577686240965, 'bagging_freq': 1, 'min_child_samples': 96}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:03,271]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:07,882]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:12,972]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:21,168]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:26,938]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:30,734]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:35,341]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:41,601]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:46,145]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:51,441]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:40:56,059]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:01,106]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:06,655]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:11,262]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:48,687]\u001b[0m Trial 244 finished with value: 0.9444550687498805 and parameters: {'lambda_l1': 0.0002036623168757251, 'lambda_l2': 0.006006535254678557, 'num_leaves': 215, 'feature_fraction': 0.9253229899202093, 'bagging_fraction': 0.6880698265447902, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:54,485]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:41:58,496]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:05,675]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:16,623]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 27.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:21,269]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:26,960]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:33,177]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:39,276]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:43,691]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:48,271]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:54,532]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:42:58,714]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:05,669]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:09,875]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:15,085]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:20,144]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:23,667]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:30,759]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:34,913]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:40,565]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:47,697]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 17.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:53,115]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:43:57,659]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:02,170]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:07,650]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:12,015]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:21,808]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 25.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:26,653]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:32,936]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:36,245]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:41,807]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:46,993]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 19:44:52,300]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:44:57,150]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:45:34,096]\u001b[0m Trial 279 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.00031112512112930536, 'lambda_l2': 0.0001880477331732333, 'num_leaves': 220, 'feature_fraction': 0.852594454408188, 'bagging_fraction': 0.6470457333056485, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:45:38,422]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:15,718]\u001b[0m Trial 281 finished with value: 0.9430685968904783 and parameters: {'lambda_l1': 4.45254298816343e-08, 'lambda_l2': 0.001188876991401564, 'num_leaves': 227, 'feature_fraction': 0.7761533930785017, 'bagging_fraction': 0.6795762898644894, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:21,752]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:26,097]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:31,317]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:36,020]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:41,005]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:48,527]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:54,368]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:46:59,084]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:03,641]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:09,582]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:13,893]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:33,116]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 57.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:36,632]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:41,787]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:46,895]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:49,470]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:54,857]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 19:47:58,624]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 300\n",
"Best trial:\n",
" Value: 0.8423533686485246\n",
" Params: \n",
" lambda_l1: 6.031700696477528e-05\n",
" lambda_l2: 0.24394450720600092\n",
" num_leaves: 33\n",
" feature_fraction: 0.48861458318031864\n",
" bagging_fraction: 0.9555014696451426\n",
" bagging_freq: 1\n",
" min_child_samples: 79\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.026580 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2258\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
"[LightGBM] [Info] Start training from score -1.009534\n",
"[LightGBM] [Info] Start training from score -0.717940\n",
"[LightGBM] [Info] Start training from score -2.789075\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343939\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.445843\n",
"[100]\tvalid_0's multi_logloss: 0.377834\n",
"[150]\tvalid_0's multi_logloss: 0.341664\n",
"[200]\tvalid_0's multi_logloss: 0.316826\n",
"[250]\tvalid_0's multi_logloss: 0.296451\n",
"[300]\tvalid_0's multi_logloss: 0.281169\n",
"[350]\tvalid_0's multi_logloss: 0.265537\n",
"[400]\tvalid_0's multi_logloss: 0.254785\n",
"[450]\tvalid_0's multi_logloss: 0.243631\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[500]\tvalid_0's multi_logloss: 0.234101\n",
"[550]\tvalid_0's multi_logloss: 0.226309\n",
"[600]\tvalid_0's multi_logloss: 0.21796\n",
"[650]\tvalid_0's multi_logloss: 0.211559\n",
"[700]\tvalid_0's multi_logloss: 0.204727\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[750]\tvalid_0's multi_logloss: 0.198816\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[800]\tvalid_0's multi_logloss: 0.193023\n",
"[850]\tvalid_0's multi_logloss: 0.188199\n",
"[900]\tvalid_0's multi_logloss: 0.184119\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[950]\tvalid_0's multi_logloss: 0.180033\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1000]\tvalid_0's multi_logloss: 0.176498\n",
"[1050]\tvalid_0's multi_logloss: 0.172741\n",
"[1100]\tvalid_0's multi_logloss: 0.169305\n",
"[1150]\tvalid_0's multi_logloss: 0.166002\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1200]\tvalid_0's multi_logloss: 0.162865\n",
"[1250]\tvalid_0's multi_logloss: 0.159812\n",
"[1300]\tvalid_0's multi_logloss: 0.157137\n",
"[1350]\tvalid_0's multi_logloss: 0.154434\n",
"[1400]\tvalid_0's multi_logloss: 0.151964\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1450]\tvalid_0's multi_logloss: 0.149897\n",
"[1500]\tvalid_0's multi_logloss: 0.14801\n",
"[1550]\tvalid_0's multi_logloss: 0.146389\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1600]\tvalid_0's multi_logloss: 0.144423\n",
"[1650]\tvalid_0's multi_logloss: 0.142452\n",
"[1700]\tvalid_0's multi_logloss: 0.140879\n",
"[1750]\tvalid_0's multi_logloss: 0.138966\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1800]\tvalid_0's multi_logloss: 0.137507\n",
"[1850]\tvalid_0's multi_logloss: 0.135709\n",
"[1900]\tvalid_0's multi_logloss: 0.134087\n",
"[1950]\tvalid_0's multi_logloss: 0.13268\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2000]\tvalid_0's multi_logloss: 0.13141\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2050]\tvalid_0's multi_logloss: 0.130194\n",
"[2100]\tvalid_0's multi_logloss: 0.129083\n",
"[2150]\tvalid_0's multi_logloss: 0.127913\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2200]\tvalid_0's multi_logloss: 0.126733\n",
"[2250]\tvalid_0's multi_logloss: 0.125701\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2300]\tvalid_0's multi_logloss: 0.124613\n",
"[2350]\tvalid_0's multi_logloss: 0.12328\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2400]\tvalid_0's multi_logloss: 0.122431\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2450]\tvalid_0's multi_logloss: 0.121605\n",
"[2500]\tvalid_0's multi_logloss: 0.120838\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2550]\tvalid_0's multi_logloss: 0.119876\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2600]\tvalid_0's multi_logloss: 0.119193\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2650]\tvalid_0's multi_logloss: 0.118348\n",
"[2700]\tvalid_0's multi_logloss: 0.117563\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2750]\tvalid_0's multi_logloss: 0.116756\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2800]\tvalid_0's multi_logloss: 0.116241\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2850]\tvalid_0's multi_logloss: 0.11561\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2900]\tvalid_0's multi_logloss: 0.115103\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2950]\tvalid_0's multi_logloss: 0.114529\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3000]\tvalid_0's multi_logloss: 0.113988\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3050]\tvalid_0's multi_logloss: 0.113383\n",
"[3100]\tvalid_0's multi_logloss: 0.112689\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3150]\tvalid_0's multi_logloss: 0.112138\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3200]\tvalid_0's multi_logloss: 0.111635\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3250]\tvalid_0's multi_logloss: 0.111011\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3300]\tvalid_0's multi_logloss: 0.110546\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[3350]\tvalid_0's multi_logloss: 0.110069\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3400]\tvalid_0's multi_logloss: 0.109636\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3450]\tvalid_0's multi_logloss: 0.109215\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3500]\tvalid_0's multi_logloss: 0.108815\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3550]\tvalid_0's multi_logloss: 0.108466\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3600]\tvalid_0's multi_logloss: 0.10805\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3650]\tvalid_0's multi_logloss: 0.107592\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3700]\tvalid_0's multi_logloss: 0.107273\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3750]\tvalid_0's multi_logloss: 0.106966\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3800]\tvalid_0's multi_logloss: 0.106652\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3850]\tvalid_0's multi_logloss: 0.106292\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3900]\tvalid_0's multi_logloss: 0.105884\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3950]\tvalid_0's multi_logloss: 0.105564\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4000]\tvalid_0's multi_logloss: 0.105233\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4050]\tvalid_0's multi_logloss: 0.104994\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4100]\tvalid_0's multi_logloss: 0.104723\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4150]\tvalid_0's multi_logloss: 0.104496\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4200]\tvalid_0's multi_logloss: 0.104218\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4250]\tvalid_0's multi_logloss: 0.103959\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4300]\tvalid_0's multi_logloss: 0.103701\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4350]\tvalid_0's multi_logloss: 0.103406\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4400]\tvalid_0's multi_logloss: 0.103227\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4450]\tvalid_0's multi_logloss: 0.102986\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4500]\tvalid_0's multi_logloss: 0.102788\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4550]\tvalid_0's multi_logloss: 0.102466\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4600]\tvalid_0's multi_logloss: 0.102287\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4650]\tvalid_0's multi_logloss: 0.102138\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4700]\tvalid_0's multi_logloss: 0.101951\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4750]\tvalid_0's multi_logloss: 0.101735\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4800]\tvalid_0's multi_logloss: 0.101611\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4850]\tvalid_0's multi_logloss: 0.101362\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4900]\tvalid_0's multi_logloss: 0.101221\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4950]\tvalid_0's multi_logloss: 0.101075\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[5000]\tvalid_0's multi_logloss: 0.10093\n",
"Did not meet early stopping. Best iteration is:\n",
"[4988]\tvalid_0's multi_logloss: 0.100912\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 20:03:19,270]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:24,024]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:29,003]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:33,769]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:39,428]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:43,005]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:49,279]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:53,868]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:03:59,420]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:04:04,308]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:04:41,514]\u001b[0m Trial 310 finished with value: 0.9448184199957927 and parameters: {'lambda_l1': 6.33710851183545e-08, 'lambda_l2': 0.00011828921524358206, 'num_leaves': 221, 'feature_fraction': 0.7490482980682972, 'bagging_fraction': 0.7149335066227929, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:04:47,751]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:04:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:03,194]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:08,173]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:13,677]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:18,274]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:24,488]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:29,197]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:33,932]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:39,317]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:43,765]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:50,484]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:05:54,773]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:06:03,444]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 18.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:06:32,442]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 95.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:06:37,822]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:06:42,014]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:21,450]\u001b[0m Trial 328 finished with value: 0.9455738081122946 and parameters: {'lambda_l1': 3.217868638779184e-05, 'lambda_l2': 5.750590102813345e-05, 'num_leaves': 226, 'feature_fraction': 0.7826114441623828, 'bagging_fraction': 0.6679740332158755, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:27,179]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:32,088]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:37,170]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:42,279]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:46,023]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:53,069]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:07:57,424]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:25,834]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 78.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:30,146]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:35,020]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:41,479]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:45,294]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:51,071]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:08:56,090]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:01,151]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:05,433]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:42,571]\u001b[0m Trial 345 finished with value: 0.9442255837524621 and parameters: {'lambda_l1': 1.4659439917902066e-06, 'lambda_l2': 9.096938742062834e-06, 'num_leaves': 223, 'feature_fraction': 0.78931856475032, 'bagging_fraction': 0.687754997409419, 'bagging_freq': 1, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:47,319]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:52,425]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:09:57,385]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:01,115]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:07,462]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:11,891]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:17,813]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:22,777]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:28,182]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:34,019]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:38,990]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:49,735]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 31.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:55,530]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:10:59,834]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:11:04,538]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:11:09,697]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:11:47,302]\u001b[0m Trial 362 finished with value: 0.9451052762425657 and parameters: {'lambda_l1': 0.17345143397850926, 'lambda_l2': 1.530418515864943e-05, 'num_leaves': 227, 'feature_fraction': 0.7794541612455614, 'bagging_fraction': 0.8921910315517494, 'bagging_freq': 1, 'min_child_samples': 71}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:11:52,146]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:11:56,135]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:00,790]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:05,348]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:10,259]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:14,597]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:21,394]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:25,610]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:31,248]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:35,702]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:41,410]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:49,192]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:54,613]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:12:59,016]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 20:13:20,403]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 65.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:13:25,274]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:13:31,727]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:13:36,235]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:13,363]\u001b[0m Trial 381 finished with value: 0.9444646306247729 and parameters: {'lambda_l1': 0.00013491606868735477, 'lambda_l2': 2.4016424269924617e-06, 'num_leaves': 224, 'feature_fraction': 0.7645040960635633, 'bagging_fraction': 0.6776788045685024, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:21,686]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 17.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:26,224]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:30,257]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:35,262]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:40,837]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:14:46,456]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:23,408]\u001b[0m Trial 388 finished with value: 0.9461570824807328 and parameters: {'lambda_l1': 0.03582500455746271, 'lambda_l2': 1.361737608270606e-06, 'num_leaves': 236, 'feature_fraction': 0.7947192105026746, 'bagging_fraction': 0.7121032476222902, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:29,081]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:35,152]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:40,204]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:45,684]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:49,921]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:15:54,703]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:16:00,737]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:16:05,612]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:16:09,888]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:16:14,330]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:16:19,744]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 400\n",
"Best trial:\n",
" Value: 0.8423533686485246\n",
" Params: \n",
" lambda_l1: 6.031700696477528e-05\n",
" lambda_l2: 0.24394450720600092\n",
" num_leaves: 33\n",
" feature_fraction: 0.48861458318031864\n",
" bagging_fraction: 0.9555014696451426\n",
" bagging_freq: 1\n",
" min_child_samples: 79\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027687 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2260\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
"[LightGBM] [Info] Start training from score -1.009528\n",
"[LightGBM] [Info] Start training from score -0.717945\n",
"[LightGBM] [Info] Start training from score -2.789114\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343871\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.44539\n",
"[100]\tvalid_0's multi_logloss: 0.378913\n",
"[150]\tvalid_0's multi_logloss: 0.340888\n",
"[200]\tvalid_0's multi_logloss: 0.31566\n",
"[250]\tvalid_0's multi_logloss: 0.29524\n",
"[300]\tvalid_0's multi_logloss: 0.278682\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[350]\tvalid_0's multi_logloss: 0.263956\n",
"[400]\tvalid_0's multi_logloss: 0.252259\n",
"[450]\tvalid_0's multi_logloss: 0.241152\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[500]\tvalid_0's multi_logloss: 0.231783\n",
"[550]\tvalid_0's multi_logloss: 0.223767\n",
"[600]\tvalid_0's multi_logloss: 0.215283\n",
"[650]\tvalid_0's multi_logloss: 0.207762\n",
"[700]\tvalid_0's multi_logloss: 0.201304\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[750]\tvalid_0's multi_logloss: 0.19581\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[800]\tvalid_0's multi_logloss: 0.190409\n",
"[850]\tvalid_0's multi_logloss: 0.185526\n",
"[900]\tvalid_0's multi_logloss: 0.18142\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[950]\tvalid_0's multi_logloss: 0.177042\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1000]\tvalid_0's multi_logloss: 0.173195\n",
"[1050]\tvalid_0's multi_logloss: 0.168927\n",
"[1100]\tvalid_0's multi_logloss: 0.165216\n",
"[1150]\tvalid_0's multi_logloss: 0.16232\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1200]\tvalid_0's multi_logloss: 0.159091\n",
"[1250]\tvalid_0's multi_logloss: 0.156049\n",
"[1300]\tvalid_0's multi_logloss: 0.153303\n",
"[1350]\tvalid_0's multi_logloss: 0.150912\n",
"[1400]\tvalid_0's multi_logloss: 0.148654\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1450]\tvalid_0's multi_logloss: 0.1463\n",
"[1500]\tvalid_0's multi_logloss: 0.144477\n",
"[1550]\tvalid_0's multi_logloss: 0.142652\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1600]\tvalid_0's multi_logloss: 0.140806\n",
"[1650]\tvalid_0's multi_logloss: 0.138759\n",
"[1700]\tvalid_0's multi_logloss: 0.137062\n",
"[1750]\tvalid_0's multi_logloss: 0.13547\n",
"[1800]\tvalid_0's multi_logloss: 0.134167\n",
"[1850]\tvalid_0's multi_logloss: 0.132412\n",
"[1900]\tvalid_0's multi_logloss: 0.13077\n",
"[1950]\tvalid_0's multi_logloss: 0.129542\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2000]\tvalid_0's multi_logloss: 0.128147\n",
"[2050]\tvalid_0's multi_logloss: 0.126764\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2100]\tvalid_0's multi_logloss: 0.125437\n",
"[2150]\tvalid_0's multi_logloss: 0.124289\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2200]\tvalid_0's multi_logloss: 0.123232\n",
"[2250]\tvalid_0's multi_logloss: 0.122371\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2300]\tvalid_0's multi_logloss: 0.121494\n",
"[2350]\tvalid_0's multi_logloss: 0.120514\n",
"[2400]\tvalid_0's multi_logloss: 0.119568\n",
"[2450]\tvalid_0's multi_logloss: 0.118828\n",
"[2500]\tvalid_0's multi_logloss: 0.117923\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2550]\tvalid_0's multi_logloss: 0.116858\n",
"[2600]\tvalid_0's multi_logloss: 0.116208\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2650]\tvalid_0's multi_logloss: 0.115557\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2700]\tvalid_0's multi_logloss: 0.114849\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2750]\tvalid_0's multi_logloss: 0.11405\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2800]\tvalid_0's multi_logloss: 0.113411\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2850]\tvalid_0's multi_logloss: 0.112686\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2900]\tvalid_0's multi_logloss: 0.11198\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2950]\tvalid_0's multi_logloss: 0.111297\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3000]\tvalid_0's multi_logloss: 0.11087\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3050]\tvalid_0's multi_logloss: 0.110211\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3100]\tvalid_0's multi_logloss: 0.109625\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3150]\tvalid_0's multi_logloss: 0.109116\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3200]\tvalid_0's multi_logloss: 0.108663\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3250]\tvalid_0's multi_logloss: 0.108126\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3300]\tvalid_0's multi_logloss: 0.107707\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3350]\tvalid_0's multi_logloss: 0.107195\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3400]\tvalid_0's multi_logloss: 0.106732\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3450]\tvalid_0's multi_logloss: 0.106219\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3500]\tvalid_0's multi_logloss: 0.105908\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3550]\tvalid_0's multi_logloss: 0.105555\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3600]\tvalid_0's multi_logloss: 0.105145\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3650]\tvalid_0's multi_logloss: 0.104808\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3700]\tvalid_0's multi_logloss: 0.104423\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3750]\tvalid_0's multi_logloss: 0.104159\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3800]\tvalid_0's multi_logloss: 0.103877\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3850]\tvalid_0's multi_logloss: 0.103484\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3900]\tvalid_0's multi_logloss: 0.103104\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3950]\tvalid_0's multi_logloss: 0.102836\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4000]\tvalid_0's multi_logloss: 0.102578\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4050]\tvalid_0's multi_logloss: 0.102273\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4100]\tvalid_0's multi_logloss: 0.101997\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4150]\tvalid_0's multi_logloss: 0.101719\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4200]\tvalid_0's multi_logloss: 0.101443\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4250]\tvalid_0's multi_logloss: 0.101108\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4300]\tvalid_0's multi_logloss: 0.100871\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4350]\tvalid_0's multi_logloss: 0.10061\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4400]\tvalid_0's multi_logloss: 0.100397\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4450]\tvalid_0's multi_logloss: 0.100218\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4500]\tvalid_0's multi_logloss: 0.100004\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4550]\tvalid_0's multi_logloss: 0.0997785\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4600]\tvalid_0's multi_logloss: 0.0996502\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4650]\tvalid_0's multi_logloss: 0.0993899\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4700]\tvalid_0's multi_logloss: 0.0991771\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4750]\tvalid_0's multi_logloss: 0.0989407\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4800]\tvalid_0's multi_logloss: 0.0987919\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4850]\tvalid_0's multi_logloss: 0.0985558\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4900]\tvalid_0's multi_logloss: 0.0983888\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4950]\tvalid_0's multi_logloss: 0.098254\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[5000]\tvalid_0's multi_logloss: 0.0981239\n",
"Did not meet early stopping. Best iteration is:\n",
"[5000]\tvalid_0's multi_logloss: 0.0981239\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 20:31:41,953]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:31:46,431]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:31:50,256]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:31:56,970]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:03,528]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:07,953]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:13,149]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:18,395]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:23,606]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:29,333]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:32:33,493]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:33:10,283]\u001b[0m Trial 411 finished with value: 0.9432215868887571 and parameters: {'lambda_l1': 1.0106430243212002e-07, 'lambda_l2': 1.7338808138171514e-05, 'num_leaves': 211, 'feature_fraction': 0.8525200092071654, 'bagging_fraction': 0.71247365704883, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:33:15,221]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:33:20,036]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:33:32,811]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 35.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:33:36,360]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:13,257]\u001b[0m Trial 416 finished with value: 0.944780172496223 and parameters: {'lambda_l1': 6.61840029683868e-08, 'lambda_l2': 6.189598777012943e-05, 'num_leaves': 229, 'feature_fraction': 0.7543055044592567, 'bagging_fraction': 0.942324205989867, 'bagging_freq': 1, 'min_child_samples': 88}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:18,909]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:23,665]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:29,134]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:33,970]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:39,344]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:44,235]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:47,811]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:53,765]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:34:59,389]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:04,937]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:10,624]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:16,512]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:22,165]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:27,930]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:33,197]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:38,098]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:43,682]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:48,031]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:53,726]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:35:58,420]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:03,775]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:09,687]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:14,370]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:19,980]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:23,347]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:30,224]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:34,705]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:39,922]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:45,883]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:49,825]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:36:56,073]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:00,642]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:05,855]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:10,406]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:15,960]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:22,067]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:26,480]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:31,630]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:36,056]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:40,816]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:46,124]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:50,577]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:37:58,422]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:02,782]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:09,254]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:13,847]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:19,221]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:23,415]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:28,284]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:33,838]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:38,296]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:43,969]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:48,921]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:54,443]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:38:59,568]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:04,433]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:10,663]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:14,889]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:20,884]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:25,563]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:30,567]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:36,476]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:39,231]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:44,485]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:49,432]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:39:54,124]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m[I 2021-09-07 20:39:58,219]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:40:02,806]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:40:09,290]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:40:49,570]\u001b[0m Trial 486 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 0.00020397245265135663, 'lambda_l2': 9.093796518313558e-06, 'num_leaves': 214, 'feature_fraction': 0.8663304164529336, 'bagging_fraction': 0.6476610679220174, 'bagging_freq': 4, 'min_child_samples': 87}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:40:54,931]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:00,439]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:05,860]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:10,637]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:15,141]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:19,440]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:23,700]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:28,019]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:33,694]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:51,434]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 56.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:41:57,024]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:42:02,000]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
"\u001b[32m[I 2021-09-07 20:42:07,346]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of finished trials: 500\n",
"Best trial:\n",
" Value: 0.8423533686485246\n",
" Params: \n",
" lambda_l1: 6.031700696477528e-05\n",
" lambda_l2: 0.24394450720600092\n",
" num_leaves: 33\n",
" feature_fraction: 0.48861458318031864\n",
" bagging_fraction: 0.9555014696451426\n",
" bagging_freq: 1\n",
" min_child_samples: 79\n",
"[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.025451 seconds.\n",
"You can set `force_col_wise=true` to remove the overhead.\n",
"[LightGBM] [Info] Total Bins 2268\n",
"[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
"[LightGBM] [Info] Start training from score -1.009528\n",
"[LightGBM] [Info] Start training from score -0.717945\n",
"[LightGBM] [Info] Start training from score -2.789114\n",
"[LightGBM] [Info] Start training from score -5.348131\n",
"[LightGBM] [Info] Start training from score -4.112163\n",
"[LightGBM] [Info] Start training from score -3.508220\n",
"[LightGBM] [Info] Start training from score -3.343871\n",
"Training until validation scores don't improve for 20 rounds\n",
"[50]\tvalid_0's multi_logloss: 0.44195\n",
"[100]\tvalid_0's multi_logloss: 0.375935\n",
"[150]\tvalid_0's multi_logloss: 0.338811\n",
"[200]\tvalid_0's multi_logloss: 0.313199\n",
"[250]\tvalid_0's multi_logloss: 0.294071\n",
"[300]\tvalid_0's multi_logloss: 0.277107\n",
"[350]\tvalid_0's multi_logloss: 0.264836\n",
"[400]\tvalid_0's multi_logloss: 0.252349\n",
"[450]\tvalid_0's multi_logloss: 0.241412\n",
"[500]\tvalid_0's multi_logloss: 0.231972\n",
"[550]\tvalid_0's multi_logloss: 0.22405\n",
"[600]\tvalid_0's multi_logloss: 0.217065\n",
"[650]\tvalid_0's multi_logloss: 0.210183\n",
"[700]\tvalid_0's multi_logloss: 0.204259\n",
"[750]\tvalid_0's multi_logloss: 0.198327\n",
"[800]\tvalid_0's multi_logloss: 0.192336\n",
"[850]\tvalid_0's multi_logloss: 0.187595\n",
"[900]\tvalid_0's multi_logloss: 0.183324\n",
"[950]\tvalid_0's multi_logloss: 0.178581\n",
"[1000]\tvalid_0's multi_logloss: 0.174766\n",
"[1050]\tvalid_0's multi_logloss: 0.171164\n",
"[1100]\tvalid_0's multi_logloss: 0.167795\n",
"[1150]\tvalid_0's multi_logloss: 0.164106\n",
"[1200]\tvalid_0's multi_logloss: 0.160688\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1250]\tvalid_0's multi_logloss: 0.157639\n",
"[1300]\tvalid_0's multi_logloss: 0.155048\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1350]\tvalid_0's multi_logloss: 0.152564\n",
"[1400]\tvalid_0's multi_logloss: 0.14986\n",
"[1450]\tvalid_0's multi_logloss: 0.147446\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1500]\tvalid_0's multi_logloss: 0.145395\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1550]\tvalid_0's multi_logloss: 0.143255\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1600]\tvalid_0's multi_logloss: 0.141166\n",
"[1650]\tvalid_0's multi_logloss: 0.139344\n",
"[1700]\tvalid_0's multi_logloss: 0.137731\n",
"[1750]\tvalid_0's multi_logloss: 0.136312\n",
"[1800]\tvalid_0's multi_logloss: 0.134798\n",
"[1850]\tvalid_0's multi_logloss: 0.133435\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[1900]\tvalid_0's multi_logloss: 0.131893\n",
"[1950]\tvalid_0's multi_logloss: 0.130715\n",
"[2000]\tvalid_0's multi_logloss: 0.129298\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2050]\tvalid_0's multi_logloss: 0.127718\n",
"[2100]\tvalid_0's multi_logloss: 0.126323\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2150]\tvalid_0's multi_logloss: 0.125164\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2200]\tvalid_0's multi_logloss: 0.124135\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2250]\tvalid_0's multi_logloss: 0.123227\n",
"[2300]\tvalid_0's multi_logloss: 0.122213\n",
"[2350]\tvalid_0's multi_logloss: 0.121182\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2400]\tvalid_0's multi_logloss: 0.120412\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2450]\tvalid_0's multi_logloss: 0.119649\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2500]\tvalid_0's multi_logloss: 0.118669\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2550]\tvalid_0's multi_logloss: 0.11796\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2600]\tvalid_0's multi_logloss: 0.11723\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2650]\tvalid_0's multi_logloss: 0.116418\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2700]\tvalid_0's multi_logloss: 0.11571\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2750]\tvalid_0's multi_logloss: 0.114968\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2800]\tvalid_0's multi_logloss: 0.114305\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2850]\tvalid_0's multi_logloss: 0.113767\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[2900]\tvalid_0's multi_logloss: 0.113049\n",
"[2950]\tvalid_0's multi_logloss: 0.112492\n",
"[3000]\tvalid_0's multi_logloss: 0.111726\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3050]\tvalid_0's multi_logloss: 0.111202\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3100]\tvalid_0's multi_logloss: 0.110604\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3150]\tvalid_0's multi_logloss: 0.110046\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3200]\tvalid_0's multi_logloss: 0.109533\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3250]\tvalid_0's multi_logloss: 0.109078\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3300]\tvalid_0's multi_logloss: 0.108525\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3350]\tvalid_0's multi_logloss: 0.108166\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3400]\tvalid_0's multi_logloss: 0.107835\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3450]\tvalid_0's multi_logloss: 0.107511\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3500]\tvalid_0's multi_logloss: 0.107145\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3550]\tvalid_0's multi_logloss: 0.106771\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3600]\tvalid_0's multi_logloss: 0.106495\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3650]\tvalid_0's multi_logloss: 0.106142\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3700]\tvalid_0's multi_logloss: 0.105776\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3750]\tvalid_0's multi_logloss: 0.105335\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3800]\tvalid_0's multi_logloss: 0.104977\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3850]\tvalid_0's multi_logloss: 0.104609\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3900]\tvalid_0's multi_logloss: 0.104318\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[3950]\tvalid_0's multi_logloss: 0.104044\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4000]\tvalid_0's multi_logloss: 0.10378\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4050]\tvalid_0's multi_logloss: 0.103526\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4100]\tvalid_0's multi_logloss: 0.103203\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4150]\tvalid_0's multi_logloss: 0.10292\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4200]\tvalid_0's multi_logloss: 0.102592\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4250]\tvalid_0's multi_logloss: 0.102358\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4300]\tvalid_0's multi_logloss: 0.102136\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4350]\tvalid_0's multi_logloss: 0.101919\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4400]\tvalid_0's multi_logloss: 0.101602\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4450]\tvalid_0's multi_logloss: 0.101265\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4500]\tvalid_0's multi_logloss: 0.10099\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4550]\tvalid_0's multi_logloss: 0.100813\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4600]\tvalid_0's multi_logloss: 0.100597\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4650]\tvalid_0's multi_logloss: 0.100381\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4700]\tvalid_0's multi_logloss: 0.100161\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4750]\tvalid_0's multi_logloss: 0.100034\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4800]\tvalid_0's multi_logloss: 0.0998763\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4850]\tvalid_0's multi_logloss: 0.0996593\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4900]\tvalid_0's multi_logloss: 0.099492\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[4950]\tvalid_0's multi_logloss: 0.0993946\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
"[5000]\tvalid_0's multi_logloss: 0.099238\n",
"Did not meet early stopping. Best iteration is:\n",
"[5000]\tvalid_0's multi_logloss: 0.099238\n",
"Wall time: 2h 35min 57s\n"
]
}
],
"source": [
"%%time\n",
"if __name__ == \"__main__\":\n",
" test_predict = 0\n",
" study = optuna.create_study(\n",
" pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\" # 指定是越小越好\n",
" )\n",
" for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
" study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n",
" train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n",
"\n",
" print(\"Number of finished trials: {}\".format(len(study.trials)))\n",
"\n",
" print(\"Best trial:\")\n",
" trial = study.best_trial\n",
"\n",
" print(\" Value: {}\".format(trial.value))\n",
"\n",
" print(\" Params: \")\n",
" for key, value in trial.params.items():\n",
" print(\" {}: {}\".format(key, value))\n",
" \n",
" params = {\"boosting_type\": \"gbdt\",\n",
" \"objective\": \"multiclass\",\n",
" \"num_class\": 7,\n",
" \"metric\": \"multi_logloss\"}\n",
" for key, value in trial.params.items():\n",
" params[key]=value\n",
" \n",
" dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
" dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
" model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
" verbose_eval=50,\n",
" early_stopping_rounds=20,\n",
" num_boost_round=5000)\n",
" test_predict += model.predict(test_x)\n",
" predict = test_predict/5"
]
},
{
"cell_type": "code",
"execution_count": 73,
"id": "d0cad2c6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"使用optuna的多分类分数： 0.968056177067915\n"
]
}
],
"source": [
"# 评估指标acc，越大越好\n",
"print('使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))"
]
},
{
"cell_type": "markdown",
"id": "63f90cf8",
"metadata": {},
"source": [
"### 分类任务的结论\n",
"不使用optuna的ACC分数是0.8631716636260369，使用的ACC分数是0.968056177067915，提升了0.1048845134418781。\n",
"\n",
"结合Stacking.ipynb里的单模随机森林最好分数0.9535087055000585甚至Stacking后的融合分数0.9589887988544127进行对比，也都是上涨的。"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in new issue View Git Blame Copy Permalink