{ "cells": [ { "cell_type": "markdown", "id": "b3d4d608", "metadata": {}, "source": [ "# Optuna一种超参数优化框架\n", "https://github.com/optuna/optuna\n", "\n", "分别采用两组数据集进行比较，分别是加利福尼亚住房数据集（回归）和森林植被类型（多分类）" ] }, { "cell_type": "markdown", "id": "b22b6f90", "metadata": {}, "source": [ "## 结论：效果不错" ] }, { "cell_type": "markdown", "id": "a2d41e62", "metadata": {}, "source": [ "## 波士顿房价预测任务（回归）" ] }, { "cell_type": "code", "execution_count": 1, "id": "f5e0e977", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import time\n", "import gc\n", "\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import mean_squared_error\n", "import lightgbm as lgb # 使用lgb模型" ] }, { "cell_type": "code", "execution_count": 2, "id": "5eda2637", "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import fetch_california_housing\n", "data = fetch_california_housing()\n", "X, y = data['data'], data['target']" ] }, { "cell_type": "code", "execution_count": 3, "id": "722beb8e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MedIncHouseAgeAveRoomsAveBedrmsPopulationAveOccupLatitudeLongitude
08.325241.06.9841271.02381322.02.55555637.88-122.23
18.301421.06.2381370.971882401.02.10984237.86-122.22
\n", "
" ], "text/plain": [ " MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude \\\n", "0 8.3252 41.0 6.984127 1.02381 322.0 2.555556 37.88 \n", "1 8.3014 21.0 6.238137 0.97188 2401.0 2.109842 37.86 \n", "\n", " Longitude \n", "0 -122.23 \n", "1 -122.22 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X = pd.DataFrame(X,columns=data.feature_names)\n", "X.head(2)" ] }, { "cell_type": "code", "execution_count": 4, "id": "08ebab89", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 20640 entries, 0 to 20639\n", "Data columns (total 8 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 MedInc 20640 non-null float64\n", " 1 HouseAge 20640 non-null float64\n", " 2 AveRooms 20640 non-null float64\n", " 3 AveBedrms 20640 non-null float64\n", " 4 Population 20640 non-null float64\n", " 5 AveOccup 20640 non-null float64\n", " 6 Latitude 20640 non-null float64\n", " 7 Longitude 20640 non-null float64\n", "dtypes: float64(8)\n", "memory usage: 1.3 MB\n" ] } ], "source": [ "X.info()" ] }, { "cell_type": "code", "execution_count": 5, "id": "8932c66c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "训练集： (18576, 8)\n", "测试集： (2064, 8)\n" ] } ], "source": [ "# 切分训练和测试集\n", "train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n", "train_y = pd.DataFrame(train_y,columns=['result'])\n", "test_y = pd.DataFrame(test_y,columns=['result'])\n", "\n", "print('训练集：',train_x.shape)\n", "print('测试集：', test_x.shape)" ] }, { "cell_type": "code", "execution_count": 6, "id": "94561283", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import KFold\n", "skf = KFold(n_splits=5, shuffle=True, random_state=42)" ] }, { "cell_type": "markdown", "id": "e87a2121", "metadata": {}, "source": [ "### 使用LGB作为模型并CV，不使用optuna调参" ] }, { "cell_type": "code", "execution_count": 7, "id": "5cad8967", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000866 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1837\n", "[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.073247\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.492877\n", "[100]\tvalid_0's rmse: 0.469235\n", "[150]\tvalid_0's rmse: 0.460357\n", "[200]\tvalid_0's rmse: 0.455302\n", "[250]\tvalid_0's rmse: 0.452674\n", "[300]\tvalid_0's rmse: 0.451179\n", "Early stopping, best iteration is:\n", "[313]\tvalid_0's rmse: 0.450303\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000494 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.070396\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.496484\n", "[100]\tvalid_0's rmse: 0.476225\n", "[150]\tvalid_0's rmse: 0.469582\n", "[200]\tvalid_0's rmse: 0.464643\n", "[250]\tvalid_0's rmse: 0.463244\n", "[300]\tvalid_0's rmse: 0.461886\n", "[350]\tvalid_0's rmse: 0.460305\n", "[400]\tvalid_0's rmse: 0.4596\n", "[450]\tvalid_0's rmse: 0.458645\n", "Early stopping, best iteration is:\n", "[435]\tvalid_0's rmse: 0.458368\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000800 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1837\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.069154\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.48676\n", "[100]\tvalid_0's rmse: 0.466911\n", "[150]\tvalid_0's rmse: 0.459693\n", "[200]\tvalid_0's rmse: 0.457631\n", "[250]\tvalid_0's rmse: 0.452574\n", "Early stopping, best iteration is:\n", "[264]\tvalid_0's rmse: 0.451947\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000779 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.063366\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.492482\n", "[100]\tvalid_0's rmse: 0.468929\n", "[150]\tvalid_0's rmse: 0.459713\n", "[200]\tvalid_0's rmse: 0.456203\n", "[250]\tvalid_0's rmse: 0.454063\n", "[300]\tvalid_0's rmse: 0.451901\n", "Early stopping, best iteration is:\n", "[322]\tvalid_0's rmse: 0.450961\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000694 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.066924\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.488385\n", "[100]\tvalid_0's rmse: 0.464851\n", "[150]\tvalid_0's rmse: 0.455183\n", "[200]\tvalid_0's rmse: 0.450506\n", "[250]\tvalid_0's rmse: 0.4461\n", "[300]\tvalid_0's rmse: 0.44438\n", "[350]\tvalid_0's rmse: 0.441808\n", "[400]\tvalid_0's rmse: 0.44079\n", "Early stopping, best iteration is:\n", "[411]\tvalid_0's rmse: 0.440543\n", "Wall time: 3.44 s\n" ] } ], "source": [ "%%time\n", "test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n", "params = {'boosting_type': 'gbdt',\n", " 'objective': 'regression',\n", " \"metric\": 'rmse'}\n", "for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n", " dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n", " dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n", " model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n", " verbose_eval=50,\n", " early_stopping_rounds=20,\n", " num_boost_round=5000)\n", " test_predict[:,i] = model.predict(test_x)\n", "predict = np.mean(test_predict,axis=1)" ] }, { "cell_type": "code", "execution_count": 8, "id": "886bfdad", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.4346521330333544" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 评估指标rmse，越小越好\n", "np.sqrt(mean_squared_error(test_y,predict))" ] }, { "cell_type": "markdown", "id": "c271442c", "metadata": {}, "source": [ "### 使用LGB作为模型并CV，使用optuna调参" ] }, { "cell_type": "code", "execution_count": 9, "id": "9a87f9db", "metadata": { "scrolled": true }, "outputs": [], "source": [ "# pip install optuna" ] }, { "cell_type": "code", "execution_count": 10, "id": "d8a0279a", "metadata": {}, "outputs": [], "source": [ "import optuna" ] }, { "cell_type": "code", "execution_count": 11, "id": "7433fdcb", "metadata": {}, "outputs": [], "source": [ "def objective(trial,train_x, train_y, valid_x, valid_y):\n", " dtrain = lgb.Dataset(train_x, label=train_y)\n", " dvalid = lgb.Dataset(valid_x, label=valid_y)\n", "\n", " param = {\n", " \"objective\": \"regression\",\n", " \"metric\": \"rmse\",\n", " \"verbosity\": -1,\n", " \"boosting_type\": \"gbdt\",\n", " 'random_state':42,\n", " \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n", " \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n", " \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n", " \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n", " \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n", " \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n", " \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n", " }\n", "\n", " # Add a callback for pruning.\n", " pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"rmse\")\n", " gbm = lgb.train(\n", " param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n", " )\n", "\n", " preds = gbm.predict(valid_x)\n", " pred_labels = np.rint(preds)\n", " rmse = np.sqrt(mean_squared_error(valid_y,pred_labels))\n", " return rmse" ] }, { "cell_type": "code", "execution_count": 12, "id": "b9018adb", "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:27:35,965]\u001b[0m A new study created in memory with name: no-name-cc424a48-83c6-4329-92a3-261d08a5edb9\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:36,779]\u001b[0m Trial 0 finished with value: 0.5178928385298298 and parameters: {'lambda_l1': 0.9464949700025425, 'lambda_l2': 1.4692321446797693e-05, 'num_leaves': 135, 'feature_fraction': 0.6448685329355106, 'bagging_fraction': 0.9048970068857253, 'bagging_freq': 7, 'min_child_samples': 30}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:37,641]\u001b[0m Trial 1 finished with value: 0.5336017055299863 and parameters: {'lambda_l1': 0.6354399847075458, 'lambda_l2': 4.87700419620481, 'num_leaves': 237, 'feature_fraction': 0.895991820221385, 'bagging_fraction': 0.6704364287317786, 'bagging_freq': 3, 'min_child_samples': 49}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:38,288]\u001b[0m Trial 2 finished with value: 0.5345721353318472 and parameters: {'lambda_l1': 9.52099217128906e-05, 'lambda_l2': 1.1454590646374039e-07, 'num_leaves': 170, 'feature_fraction': 0.7899599630691868, 'bagging_fraction': 0.5059741439893897, 'bagging_freq': 2, 'min_child_samples': 50}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:38,956]\u001b[0m Trial 3 finished with value: 0.543693048566443 and parameters: {'lambda_l1': 0.1481577233973877, 'lambda_l2': 3.179336503731134e-07, 'num_leaves': 128, 'feature_fraction': 0.9783519466402901, 'bagging_fraction': 0.6889127950806107, 'bagging_freq': 5, 'min_child_samples': 37}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:39,424]\u001b[0m Trial 4 finished with value: 0.5501760993688642 and parameters: {'lambda_l1': 3.7638689664291936, 'lambda_l2': 0.020384770050374573, 'num_leaves': 175, 'feature_fraction': 0.4840841073088404, 'bagging_fraction': 0.8334033110992205, 'bagging_freq': 5, 'min_child_samples': 35}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:39,493]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:39,774]\u001b[0m Trial 6 pruned. Trial was pruned at iteration 33.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:39,828]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:40,187]\u001b[0m Trial 8 finished with value: 0.5249490156742078 and parameters: {'lambda_l1': 8.186728837628555e-05, 'lambda_l2': 2.6479736415938615, 'num_leaves': 72, 'feature_fraction': 0.7599981859922114, 'bagging_fraction': 0.9692259725506919, 'bagging_freq': 5, 'min_child_samples': 38}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:40,299]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:40,353]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:40,398]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:40,470]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:41,280]\u001b[0m Trial 13 finished with value: 0.5325142394678348 and parameters: {'lambda_l1': 0.009551573559223583, 'lambda_l2': 0.0003114297759461472, 'num_leaves': 152, 'feature_fraction': 0.8236610445865283, 'bagging_fraction': 0.8389170404125451, 'bagging_freq': 4, 'min_child_samples': 20}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:41,391]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,015]\u001b[0m Trial 15 finished with value: 0.5314302398651312 and parameters: {'lambda_l1': 0.024227381422510946, 'lambda_l2': 0.33276527010426776, 'num_leaves': 91, 'feature_fraction': 0.7874026441153832, 'bagging_fraction': 0.7899555875604796, 'bagging_freq': 7, 'min_child_samples': 19}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,091]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,154]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,226]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,338]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,384]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,829]\u001b[0m Trial 21 finished with value: 0.5245233417218278 and parameters: {'lambda_l1': 0.07296901453015396, 'lambda_l2': 0.19272462963641587, 'num_leaves': 90, 'feature_fraction': 0.752955332537125, 'bagging_fraction': 0.7944607588474272, 'bagging_freq': 7, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:42,901]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:43,287]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 97.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:43,371]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:43,489]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:43,572]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:43,632]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:44,171]\u001b[0m Trial 28 finished with value: 0.5255422897694476 and parameters: {'lambda_l1': 0.028260092955639447, 'lambda_l2': 0.08413723627697263, 'num_leaves': 74, 'feature_fraction': 0.8042469147694201, 'bagging_fraction': 0.7548217217698655, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:44,331]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:44,516]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 16.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:44,637]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:44,723]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:45,592]\u001b[0m Trial 33 finished with value: 0.5218146276752049 and parameters: {'lambda_l1': 0.0016497886750146257, 'lambda_l2': 0.21484669274687151, 'num_leaves': 133, 'feature_fraction': 0.6885034648040802, 'bagging_fraction': 0.9281320338482011, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:45,711]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:46,669]\u001b[0m Trial 35 finished with value: 0.5205542553680208 and parameters: {'lambda_l1': 4.3493248665689235e-05, 'lambda_l2': 3.680853885692289, 'num_leaves': 169, 'feature_fraction': 0.7104567429350184, 'bagging_fraction': 0.8672533271233644, 'bagging_freq': 1, 'min_child_samples': 23}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:47,676]\u001b[0m Trial 36 finished with value: 0.5206023202395521 and parameters: {'lambda_l1': 0.0006187306261428898, 'lambda_l2': 6.130428276943865e-08, 'num_leaves': 168, 'feature_fraction': 0.7036284508576037, 'bagging_fraction': 0.868428878180794, 'bagging_freq': 2, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:47,808]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:47,937]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:48,844]\u001b[0m Trial 39 finished with value: 0.5191042263340773 and parameters: {'lambda_l1': 0.0005562321404093257, 'lambda_l2': 2.232696278872472e-08, 'num_leaves': 161, 'feature_fraction': 0.710527336424428, 'bagging_fraction': 0.9309118257124477, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:49,889]\u001b[0m Trial 40 finished with value: 0.5168505162425636 and parameters: {'lambda_l1': 0.0004441190594738051, 'lambda_l2': 1.0707946566246148e-08, 'num_leaves': 190, 'feature_fraction': 0.7132737394337006, 'bagging_fraction': 0.8296082050311246, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:50,929]\u001b[0m Trial 41 finished with value: 0.5262832333602245 and parameters: {'lambda_l1': 0.0004892025280466791, 'lambda_l2': 1.89573604152611e-08, 'num_leaves': 189, 'feature_fraction': 0.7134600565845959, 'bagging_fraction': 0.8334328552244646, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:27:51,023]\u001b[0m Trial 42 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:51,805]\u001b[0m Trial 43 finished with value: 0.516065766031832 and parameters: {'lambda_l1': 0.00021869661941011055, 'lambda_l2': 1.0674672013946999e-08, 'num_leaves': 201, 'feature_fraction': 0.7039965594752932, 'bagging_fraction': 0.8165783106186105, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:51,940]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:52,044]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:52,218]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:53,067]\u001b[0m Trial 47 finished with value: 0.5256815758791472 and parameters: {'lambda_l1': 6.162602167779085e-05, 'lambda_l2': 1.6462819728079273e-07, 'num_leaves': 201, 'feature_fraction': 0.7163097853297453, 'bagging_fraction': 0.9420618119436382, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:53,178]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:53,762]\u001b[0m Trial 49 finished with value: 0.5213791636070063 and parameters: {'lambda_l1': 2.774750232046531e-07, 'lambda_l2': 2.4404405074646332e-08, 'num_leaves': 154, 'feature_fraction': 0.6882189607163384, 'bagging_fraction': 0.9871043449165129, 'bagging_freq': 1, 'min_child_samples': 9}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:53,875]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:54,587]\u001b[0m Trial 51 finished with value: 0.5174618905113623 and parameters: {'lambda_l1': 0.000381507052985562, 'lambda_l2': 4.901891994722806e-08, 'num_leaves': 166, 'feature_fraction': 0.7117978094083592, 'bagging_fraction': 0.8956613846537137, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:55,304]\u001b[0m Trial 52 finished with value: 0.5219780933938188 and parameters: {'lambda_l1': 0.0003055316771241415, 'lambda_l2': 1.0975399369852818e-08, 'num_leaves': 175, 'feature_fraction': 0.7709159871191618, 'bagging_fraction': 0.8946659725371712, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:56,481]\u001b[0m Trial 53 finished with value: 0.522019841081264 and parameters: {'lambda_l1': 3.0853077659712017e-06, 'lambda_l2': 3.167435996190143e-08, 'num_leaves': 199, 'feature_fraction': 0.700370780125537, 'bagging_fraction': 0.9659440325481246, 'bagging_freq': 1, 'min_child_samples': 5}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:57,422]\u001b[0m Trial 54 finished with value: 0.5210662969810745 and parameters: {'lambda_l1': 7.857450656938453e-05, 'lambda_l2': 4.5159973730477044e-07, 'num_leaves': 153, 'feature_fraction': 0.7263108852936352, 'bagging_fraction': 0.936630215722916, 'bagging_freq': 1, 'min_child_samples': 19}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:57,568]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:57,678]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:57,803]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:57,889]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:58,284]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:58,510]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 15.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:59,683]\u001b[0m Trial 61 finished with value: 0.5238780309760502 and parameters: {'lambda_l1': 0.0005622649168938934, 'lambda_l2': 7.041773458860283e-08, 'num_leaves': 165, 'feature_fraction': 0.6971774484774765, 'bagging_fraction': 0.8845735572162268, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:27:59,867]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:01,236]\u001b[0m Trial 63 finished with value: 0.5172018048571797 and parameters: {'lambda_l1': 4.4774427854289624e-05, 'lambda_l2': 1.0431434685856767e-07, 'num_leaves': 175, 'feature_fraction': 0.7067550902029666, 'bagging_fraction': 0.7960327498051027, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:01,369]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:01,513]\u001b[0m Trial 65 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:02,393]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 76.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:02,529]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:02,721]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:03,060]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 44.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:03,135]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:03,486]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 56.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:03,587]\u001b[0m Trial 72 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,107]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 68.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,203]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,307]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,631]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 39.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,725]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,831]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,920]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:04,992]\u001b[0m Trial 80 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:05,099]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:05,766]\u001b[0m Trial 82 finished with value: 0.5178938881576146 and parameters: {'lambda_l1': 1.2203483524616316e-08, 'lambda_l2': 5.507928324956204e-07, 'num_leaves': 185, 'feature_fraction': 0.7219415365227362, 'bagging_fraction': 0.9438920094595108, 'bagging_freq': 1, 'min_child_samples': 17}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:05,866]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:05,961]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:06,778]\u001b[0m Trial 85 finished with value: 0.5122528701381253 and parameters: {'lambda_l1': 6.371822640459341e-07, 'lambda_l2': 3.03967794068636e-08, 'num_leaves': 189, 'feature_fraction': 0.7066521328434867, 'bagging_fraction': 0.8972190570020028, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:07,709]\u001b[0m Trial 86 finished with value: 0.5128398316765018 and parameters: {'lambda_l1': 1.5059254193489838e-07, 'lambda_l2': 2.6171717314455622e-08, 'num_leaves': 192, 'feature_fraction': 0.711268040370677, 'bagging_fraction': 0.9174432395706226, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:09,411]\u001b[0m Trial 87 finished with value: 0.5191456815789754 and parameters: {'lambda_l1': 5.687878085793806e-08, 'lambda_l2': 2.6772763929664963e-08, 'num_leaves': 216, 'feature_fraction': 0.7413398142797374, 'bagging_fraction': 0.9993988673163303, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:09,813]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:12,526]\u001b[0m Trial 89 finished with value: 0.5102536883069437 and parameters: {'lambda_l1': 1.6446541932580473e-07, 'lambda_l2': 1.0734412329614426e-07, 'num_leaves': 193, 'feature_fraction': 0.8123395009389368, 'bagging_fraction': 0.9184649865458648, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:12,792]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:13,099]\u001b[0m Trial 91 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:13,456]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:13,664]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:14,351]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 45.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:14,508]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:14,828]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:14,962]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:16,197]\u001b[0m Trial 98 finished with value: 0.5171555001746302 and parameters: {'lambda_l1': 1.0103137663726183e-08, 'lambda_l2': 1.7418953701287595e-06, 'num_leaves': 203, 'feature_fraction': 0.7265189263478641, 'bagging_fraction': 0.9317931154547426, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:16,557]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 16.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 100\n", "Best trial:\n", " Value: 0.5102536883069437\n", " Params: \n", " lambda_l1: 1.6446541932580473e-07\n", " lambda_l2: 1.0734412329614426e-07\n", " num_leaves: 193\n", " feature_fraction: 0.8123395009389368\n", " bagging_fraction: 0.9184649865458648\n", " bagging_freq: 1\n", " min_child_samples: 10\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000712 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1837\n", "[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.073247\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.453164\n", "[100]\tvalid_0's rmse: 0.442958\n", "Early stopping, best iteration is:\n", "[110]\tvalid_0's rmse: 0.442157\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:17,677]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:17,796]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:17,941]\u001b[0m Trial 102 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,090]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,293]\u001b[0m Trial 104 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,412]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,524]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,647]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,797]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:18,911]\u001b[0m Trial 109 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:19,029]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:19,211]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 16.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:19,595]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 30.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:19,744]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:19,853]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,146]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 23.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,267]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,420]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,594]\u001b[0m Trial 118 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,780]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:20,942]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:21,073]\u001b[0m Trial 121 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:21,410]\u001b[0m Trial 122 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:21,601]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:21,799]\u001b[0m Trial 124 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:21,944]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,080]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,209]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,367]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,509]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,673]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,754]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:22,923]\u001b[0m Trial 132 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:23,074]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:23,442]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:23,682]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 16.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:23,824]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:24,010]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:24,160]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:24,458]\u001b[0m Trial 139 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:24,678]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:24,849]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:25,081]\u001b[0m Trial 142 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:25,256]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:25,501]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:26,208]\u001b[0m Trial 145 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:26,709]\u001b[0m Trial 146 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:26,919]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:27,534]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:27,729]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,152]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 24.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,278]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,393]\u001b[0m Trial 152 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,543]\u001b[0m Trial 153 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,653]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,757]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:28,887]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,001]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,176]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,332]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,541]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,732]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,884]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:29,997]\u001b[0m Trial 163 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,103]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,225]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,330]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,487]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,579]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,685]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,798]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,902]\u001b[0m Trial 171 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:30,999]\u001b[0m Trial 172 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,099]\u001b[0m Trial 173 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,206]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,352]\u001b[0m Trial 175 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,476]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,716]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,834]\u001b[0m Trial 178 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:31,983]\u001b[0m Trial 179 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,099]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,272]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,431]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,546]\u001b[0m Trial 183 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,656]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,763]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:32,965]\u001b[0m Trial 186 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:33,183]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:33,409]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:33,571]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:33,719]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:33,919]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:34,112]\u001b[0m Trial 192 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:34,276]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:34,426]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:34,633]\u001b[0m Trial 195 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:34,781]\u001b[0m Trial 196 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:35,002]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:35,242]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:35,408]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 200\n", "Best trial:\n", " Value: 0.5102536883069437\n", " Params: \n", " lambda_l1: 1.6446541932580473e-07\n", " lambda_l2: 1.0734412329614426e-07\n", " num_leaves: 193\n", " feature_fraction: 0.8123395009389368\n", " bagging_fraction: 0.9184649865458648\n", " bagging_freq: 1\n", " min_child_samples: 10\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000777 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.070396\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.465875\n", "Early stopping, best iteration is:\n", "[76]\tvalid_0's rmse: 0.460117\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:36,810]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:36,973]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,136]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,255]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,402]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,545]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,742]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:37,864]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,005]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,155]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,307]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,419]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,537]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,637]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,786]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:38,962]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,091]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,246]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,368]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,505]\u001b[0m Trial 219 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,637]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,737]\u001b[0m Trial 221 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,847]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:39,948]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,046]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,138]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,235]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,324]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,398]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,483]\u001b[0m Trial 229 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,621]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,779]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,860]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:40,937]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,077]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,188]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,312]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,431]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,533]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,664]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,790]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:41,902]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,009]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,110]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,233]\u001b[0m Trial 244 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,443]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,587]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,674]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,779]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,859]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:42,958]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,070]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,257]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,356]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,462]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,566]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,627]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,743]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,856]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:43,945]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,059]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,164]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,279]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,377]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,478]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,585]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,688]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,794]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:44,914]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,016]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,111]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,248]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,424]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,578]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,809]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:45,937]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,068]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,210]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,357]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,505]\u001b[0m Trial 279 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,662]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,828]\u001b[0m Trial 281 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:46,927]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:47,157]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:47,394]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:47,564]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:47,701]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:47,936]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:48,091]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:48,268]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:48,470]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:48,650]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:48,799]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:48,953]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,112]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,267]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,454]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,631]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,744]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:49,933]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 300\n", "Best trial:\n", " Value: 0.5102536883069437\n", " Params: \n", " lambda_l1: 1.6446541932580473e-07\n", " lambda_l2: 1.0734412329614426e-07\n", " num_leaves: 193\n", " feature_fraction: 0.8123395009389368\n", " bagging_fraction: 0.9184649865458648\n", " bagging_freq: 1\n", " min_child_samples: 10\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000797 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1837\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.069154\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.451573\n", "Early stopping, best iteration is:\n", "[63]\tvalid_0's rmse: 0.44707\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:28:51,004]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,159]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,315]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,412]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,543]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,686]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,842]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:51,977]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,105]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,202]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,341]\u001b[0m Trial 310 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,426]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,642]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,755]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,880]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:52,980]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,093]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,202]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,333]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,488]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,634]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,749]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,863]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:53,983]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,088]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,198]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,321]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,417]\u001b[0m Trial 328 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,486]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,630]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,711]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,838]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:54,968]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,077]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,179]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,319]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,518]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,609]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,741]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:55,903]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,024]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,140]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,425]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,577]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,697]\u001b[0m Trial 345 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,814]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:56,931]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,047]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,165]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,318]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,470]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,640]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,826]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:57,991]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:58,135]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:58,300]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:58,479]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:58,638]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:58,813]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,027]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,183]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,346]\u001b[0m Trial 362 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,510]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,627]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,792]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:28:59,954]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:00,188]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:00,393]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:00,557]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:00,707]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:00,836]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,018]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,171]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,355]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,634]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,803]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:01,949]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,110]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,225]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,409]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,574]\u001b[0m Trial 381 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,733]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:02,895]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:03,059]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:03,184]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:03,408]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:03,583]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:03,729]\u001b[0m Trial 388 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:29:03,924]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,120]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,292]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,406]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,514]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,671]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,760]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:04,879]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:05,049]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:05,188]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:05,277]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 400\n", "Best trial:\n", " Value: 0.5102536883069437\n", " Params: \n", " lambda_l1: 1.6446541932580473e-07\n", " lambda_l2: 1.0734412329614426e-07\n", " num_leaves: 193\n", " feature_fraction: 0.8123395009389368\n", " bagging_fraction: 0.9184649865458648\n", " bagging_freq: 1\n", " min_child_samples: 10\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000424 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.063366\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.458105\n", "[100]\tvalid_0's rmse: 0.449598\n", "[150]\tvalid_0's rmse: 0.449092\n", "Early stopping, best iteration is:\n", "[138]\tvalid_0's rmse: 0.448682\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:29:06,435]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:06,586]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:06,756]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:06,867]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:06,968]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,051]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,151]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,290]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,400]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,530]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,649]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,752]\u001b[0m Trial 411 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:07,880]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,002]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,140]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,264]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,434]\u001b[0m Trial 416 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,553]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,677]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,794]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:08,902]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,060]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,181]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,279]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,517]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,691]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:09,858]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,005]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,197]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,377]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,567]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,749]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:10,908]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,012]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,264]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,460]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,636]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,805]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:11,985]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,116]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,257]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,455]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,626]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,785]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:12,916]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,059]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,202]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,402]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,571]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,751]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:13,967]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:14,171]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:14,354]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:14,580]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:14,763]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:14,944]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:15,124]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:15,326]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:15,508]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:15,707]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:15,875]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,043]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,205]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,355]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,499]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,601]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,757]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:16,899]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,040]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,256]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,440]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,613]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,735]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,851]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:17,940]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,015]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,140]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,318]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,485]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,594]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,767]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:18,864]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,000]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,141]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,241]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,362]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,512]\u001b[0m Trial 486 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,651]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:19,777]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 16:29:19,892]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,026]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,152]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,291]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,385]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,508]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,644]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,766]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:20,912]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:21,093]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 16:29:21,228]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 500\n", "Best trial:\n", " Value: 0.5102536883069437\n", " Params: \n", " lambda_l1: 1.6446541932580473e-07\n", " lambda_l2: 1.0734412329614426e-07\n", " num_leaves: 193\n", " feature_fraction: 0.8123395009389368\n", " bagging_fraction: 0.9184649865458648\n", " bagging_freq: 1\n", " min_child_samples: 10\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.001110 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 1838\n", "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n", "[LightGBM] [Info] Start training from score 2.066924\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's rmse: 0.444941\n", "[100]\tvalid_0's rmse: 0.437147\n", "Early stopping, best iteration is:\n", "[90]\tvalid_0's rmse: 0.43678\n", "Wall time: 1min 46s\n" ] } ], "source": [ "%%time\n", "if __name__ == \"__main__\":\n", " test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n", " study = optuna.create_study(\n", " pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\" # 指定是越小越好\n", " )\n", " for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n", " study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n", " train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n", "\n", " print(\"Number of finished trials: {}\".format(len(study.trials)))\n", "\n", " print(\"Best trial:\")\n", " trial = study.best_trial\n", "\n", " print(\" Value: {}\".format(trial.value))\n", "\n", " print(\" Params: \")\n", " for key, value in trial.params.items():\n", " print(\" {}: {}\".format(key, value))\n", " \n", " params = {'boosting_type': 'gbdt',\n", " 'objective': 'regression',\n", " \"metric\": 'rmse'}\n", " for key, value in trial.params.items():\n", " params[key]=value\n", " \n", " dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n", " dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n", " model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n", " verbose_eval=50,\n", " early_stopping_rounds=20,\n", " num_boost_round=5000)\n", " test_predict[:,i] = model.predict(test_x)\n", " predict = np.mean(test_predict,axis=1)" ] }, { "cell_type": "code", "execution_count": 13, "id": "f28d82da", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.432341765333029" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 评估指标rmse，越小越好\n", "np.sqrt(mean_squared_error(test_y,predict))" ] }, { "cell_type": "markdown", "id": "0c7ee3c5", "metadata": {}, "source": [ "### 回归任务的结论\n", "不使用optuna的RMSE分数是0.4346521330333544，使用的RMSE分数是0.432341765333029，提升了0.0023103677003254。\n", "\n", "作者测了很多次，基本在0.003-0.002之间，感兴趣的可以多跑几次。" ] }, { "cell_type": "markdown", "id": "bb67b354", "metadata": {}, "source": [ "## 森林植被类型（多分类）" ] }, { "cell_type": "code", "execution_count": 25, "id": "108949a0", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import time\n", "import gc\n", "\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import accuracy_score\n", "from sklearn.preprocessing import OrdinalEncoder\n", "import lightgbm as lgb # 使用lgb模型" ] }, { "cell_type": "code", "execution_count": 26, "id": "36e26941", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "七分类任务，处理前： [1 2 3 4 5 6 7]\n", "[5 5 2 ... 3 3 3]\n", "七分类任务，处理后： [0. 1. 2. 3. 4. 5. 6.]\n", "[4. 4. 1. ... 2. 2. 2.]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ElevationAspectSlopeHorizontal_Distance_To_HydrologyVertical_Distance_To_HydrologyHorizontal_Distance_To_RoadwaysHillshade_9amHillshade_NoonHillshade_3pmHorizontal_Distance_To_Fire_Points...Soil_Type_30Soil_Type_31Soil_Type_32Soil_Type_33Soil_Type_34Soil_Type_35Soil_Type_36Soil_Type_37Soil_Type_38Soil_Type_39
02596.051.03.0258.00.0510.0221.0232.0148.06279.0...0.00.00.00.00.00.00.00.00.00.0
12590.056.02.0212.0-6.0390.0220.0235.0151.06225.0...0.00.00.00.00.00.00.00.00.00.0
\n", "

2 rows × 54 columns

\n", "
" ], "text/plain": [ " Elevation Aspect Slope Horizontal_Distance_To_Hydrology \\\n", "0 2596.0 51.0 3.0 258.0 \n", "1 2590.0 56.0 2.0 212.0 \n", "\n", " Vertical_Distance_To_Hydrology Horizontal_Distance_To_Roadways \\\n", "0 0.0 510.0 \n", "1 -6.0 390.0 \n", "\n", " Hillshade_9am Hillshade_Noon Hillshade_3pm \\\n", "0 221.0 232.0 148.0 \n", "1 220.0 235.0 151.0 \n", "\n", " Horizontal_Distance_To_Fire_Points ... Soil_Type_30 Soil_Type_31 \\\n", "0 6279.0 ... 0.0 0.0 \n", "1 6225.0 ... 0.0 0.0 \n", "\n", " Soil_Type_32 Soil_Type_33 Soil_Type_34 Soil_Type_35 Soil_Type_36 \\\n", "0 0.0 0.0 0.0 0.0 0.0 \n", "1 0.0 0.0 0.0 0.0 0.0 \n", "\n", " Soil_Type_37 Soil_Type_38 Soil_Type_39 \n", "0 0.0 0.0 0.0 \n", "1 0.0 0.0 0.0 \n", "\n", "[2 rows x 54 columns]" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.datasets import fetch_covtype\n", "data = fetch_covtype()\n", "X, y = data['data'], data['target']\n", "X = pd.DataFrame(X,columns=data.feature_names)\n", "print('七分类任务，处理前：',np.unique(y))\n", "print(y)\n", "ord = OrdinalEncoder()\n", "y = ord.fit_transform(y.reshape(-1, 1))\n", "y = y.reshape(-1, )\n", "print('七分类任务，处理后：',np.unique(y))\n", "print(y)\n", "X.head(2)" ] }, { "cell_type": "code", "execution_count": 15, "id": "152a8658", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 581012 entries, 0 to 581011\n", "Data columns (total 54 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 Elevation 581012 non-null float64\n", " 1 Aspect 581012 non-null float64\n", " 2 Slope 581012 non-null float64\n", " 3 Horizontal_Distance_To_Hydrology 581012 non-null float64\n", " 4 Vertical_Distance_To_Hydrology 581012 non-null float64\n", " 5 Horizontal_Distance_To_Roadways 581012 non-null float64\n", " 6 Hillshade_9am 581012 non-null float64\n", " 7 Hillshade_Noon 581012 non-null float64\n", " 8 Hillshade_3pm 581012 non-null float64\n", " 9 Horizontal_Distance_To_Fire_Points 581012 non-null float64\n", " 10 Wilderness_Area_0 581012 non-null float64\n", " 11 Wilderness_Area_1 581012 non-null float64\n", " 12 Wilderness_Area_2 581012 non-null float64\n", " 13 Wilderness_Area_3 581012 non-null float64\n", " 14 Soil_Type_0 581012 non-null float64\n", " 15 Soil_Type_1 581012 non-null float64\n", " 16 Soil_Type_2 581012 non-null float64\n", " 17 Soil_Type_3 581012 non-null float64\n", " 18 Soil_Type_4 581012 non-null float64\n", " 19 Soil_Type_5 581012 non-null float64\n", " 20 Soil_Type_6 581012 non-null float64\n", " 21 Soil_Type_7 581012 non-null float64\n", " 22 Soil_Type_8 581012 non-null float64\n", " 23 Soil_Type_9 581012 non-null float64\n", " 24 Soil_Type_10 581012 non-null float64\n", " 25 Soil_Type_11 581012 non-null float64\n", " 26 Soil_Type_12 581012 non-null float64\n", " 27 Soil_Type_13 581012 non-null float64\n", " 28 Soil_Type_14 581012 non-null float64\n", " 29 Soil_Type_15 581012 non-null float64\n", " 30 Soil_Type_16 581012 non-null float64\n", " 31 Soil_Type_17 581012 non-null float64\n", " 32 Soil_Type_18 581012 non-null float64\n", " 33 Soil_Type_19 581012 non-null float64\n", " 34 Soil_Type_20 581012 non-null float64\n", " 35 Soil_Type_21 581012 non-null float64\n", " 36 Soil_Type_22 581012 non-null float64\n", " 37 Soil_Type_23 581012 non-null float64\n", " 38 Soil_Type_24 581012 non-null float64\n", " 39 Soil_Type_25 581012 non-null float64\n", " 40 Soil_Type_26 581012 non-null float64\n", " 41 Soil_Type_27 581012 non-null float64\n", " 42 Soil_Type_28 581012 non-null float64\n", " 43 Soil_Type_29 581012 non-null float64\n", " 44 Soil_Type_30 581012 non-null float64\n", " 45 Soil_Type_31 581012 non-null float64\n", " 46 Soil_Type_32 581012 non-null float64\n", " 47 Soil_Type_33 581012 non-null float64\n", " 48 Soil_Type_34 581012 non-null float64\n", " 49 Soil_Type_35 581012 non-null float64\n", " 50 Soil_Type_36 581012 non-null float64\n", " 51 Soil_Type_37 581012 non-null float64\n", " 52 Soil_Type_38 581012 non-null float64\n", " 53 Soil_Type_39 581012 non-null float64\n", "dtypes: float64(54)\n", "memory usage: 239.4 MB\n" ] } ], "source": [ "X.info()" ] }, { "cell_type": "code", "execution_count": 27, "id": "67086c1c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "训练集： (522910, 54)\n", "测试集： (58102, 54)\n" ] } ], "source": [ "# 切分训练和测试集\n", "train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n", "train_y = pd.DataFrame(train_y,columns=['result'])\n", "test_y = pd.DataFrame(test_y,columns=['result'])\n", "\n", "print('训练集：',train_x.shape)\n", "print('测试集：', test_x.shape)" ] }, { "cell_type": "code", "execution_count": 28, "id": "88350615", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import StratifiedKFold\n", "skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)" ] }, { "cell_type": "code", "execution_count": 58, "id": "2e694b0c", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.030610 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2264\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.347629\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508300\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.424906\n", "[100]\tvalid_0's multi_logloss: 0.413801\n", "Early stopping, best iteration is:\n", "[90]\tvalid_0's multi_logloss: 0.37472\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031196 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2269\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.347629\n", "[LightGBM] [Info] Start training from score -4.112309\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.442955\n", "[100]\tvalid_0's multi_logloss: 0.396755\n", "[150]\tvalid_0's multi_logloss: 0.378273\n", "Early stopping, best iteration is:\n", "[135]\tvalid_0's multi_logloss: 0.375671\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.018390 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2260\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.431872\n", "[100]\tvalid_0's multi_logloss: 0.378335\n", "Early stopping, best iteration is:\n", "[109]\tvalid_0's multi_logloss: 0.37075\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.032231 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2262\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009528\n", "[LightGBM] [Info] Start training from score -0.717945\n", "[LightGBM] [Info] Start training from score -2.789114\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343871\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.437033\n", "[100]\tvalid_0's multi_logloss: 0.387111\n", "Early stopping, best iteration is:\n", "[109]\tvalid_0's multi_logloss: 0.380009\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031067 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2268\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009528\n", "[LightGBM] [Info] Start training from score -0.717945\n", "[LightGBM] [Info] Start training from score -2.789114\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343871\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.428647\n", "[100]\tvalid_0's multi_logloss: 0.485091\n", "Early stopping, best iteration is:\n", "[85]\tvalid_0's multi_logloss: 0.386918\n", "Wall time: 1min 53s\n" ] } ], "source": [ "%%time\n", "test_predict = 0\n", "params = {\"boosting_type\": \"gbdt\",\n", " \"objective\": \"multiclass\",\n", " \"num_class': 7,\n", " \"metric\": \"multi_logloss\"}\n", "for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n", " dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n", " dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n", " model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n", " verbose_eval=50,\n", " early_stopping_rounds=20,\n", " num_boost_round=5000)\n", " test_predict += model.predict(test_x)\n", "predict = test_predict/5" ] }, { "cell_type": "code", "execution_count": 62, "id": "6d7d4447", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "不使用optuna的多分类分数： 0.8631716636260369\n" ] } ], "source": [ "# 评估指标acc，越大越好\n", "print('不使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))" ] }, { "cell_type": "code", "execution_count": 70, "id": "f6571309", "metadata": {}, "outputs": [], "source": [ "import optuna\n", "def objective(trial,train_x, train_y, valid_x, valid_y):\n", " dtrain = lgb.Dataset(train_x, label=train_y)\n", " dvalid = lgb.Dataset(valid_x, label=valid_y)\n", "\n", " param = {\n", " \"objective\": \"multiclass\",\n", " \"metric\": \"multi_logloss\",\n", " \"verbosity\": -1,\n", " \"num_class\": 7,\n", " \"boosting_type\": \"gbdt\",\n", " 'random_state':42,\n", " \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n", " \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n", " \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n", " \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n", " \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n", " \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n", " \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n", " }\n", "\n", " # Add a callback for pruning.\n", " pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"multi_logloss\")\n", " gbm = lgb.train(\n", " param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n", " )\n", "\n", " preds = gbm.predict(valid_x)\n", " pred_labels = np.rint(preds)\n", " acc = accuracy_score(np.argmax(pred_labels, axis=1), valid_y)\n", " return acc" ] }, { "cell_type": "code", "execution_count": 72, "id": "4170d934", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 18:21:48,373]\u001b[0m A new study created in memory with name: no-name-7584aee7-d7a5-42ce-8426-7d4c7e034372\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:22:17,890]\u001b[0m Trial 0 finished with value: 0.8652253733912145 and parameters: {'lambda_l1': 0.00029350441808598135, 'lambda_l2': 0.0050140304157587625, 'num_leaves': 41, 'feature_fraction': 0.7195254545980898, 'bagging_fraction': 0.6783357374559799, 'bagging_freq': 6, 'min_child_samples': 32}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:22:40,277]\u001b[0m Trial 1 finished with value: 0.865770400260083 and parameters: {'lambda_l1': 8.073501039689362e-07, 'lambda_l2': 2.7820830665481296e-05, 'num_leaves': 41, 'feature_fraction': 0.8160512302334224, 'bagging_fraction': 0.6156688842906214, 'bagging_freq': 7, 'min_child_samples': 35}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:23:14,303]\u001b[0m Trial 2 finished with value: 0.9372358532060967 and parameters: {'lambda_l1': 1.1987047256838896e-07, 'lambda_l2': 0.0008429364945676234, 'num_leaves': 203, 'feature_fraction': 0.6245432353876832, 'bagging_fraction': 0.6680546579110309, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:23:37,365]\u001b[0m Trial 3 finished with value: 0.8423533686485246 and parameters: {'lambda_l1': 6.031700696477528e-05, 'lambda_l2': 0.24394450720600092, 'num_leaves': 33, 'feature_fraction': 0.48861458318031864, 'bagging_fraction': 0.9555014696451426, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:24:15,185]\u001b[0m Trial 4 finished with value: 0.9382876594442638 and parameters: {'lambda_l1': 0.013655390719191782, 'lambda_l2': 2.248487596614776e-07, 'num_leaves': 229, 'feature_fraction': 0.6314327078814898, 'bagging_fraction': 0.9426524187528247, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:24:18,728]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:25:04,830]\u001b[0m Trial 6 finished with value: 0.9459084737335297 and parameters: {'lambda_l1': 0.02773561511163358, 'lambda_l2': 0.30726353465031164, 'num_leaves': 238, 'feature_fraction': 0.8197796474237233, 'bagging_fraction': 0.9951387365343576, 'bagging_freq': 7, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:25:10,692]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:25:52,087]\u001b[0m Trial 8 finished with value: 0.9404582050448452 and parameters: {'lambda_l1': 2.969808205332883e-07, 'lambda_l2': 3.0108163042597293e-05, 'num_leaves': 184, 'feature_fraction': 0.9858004420758661, 'bagging_fraction': 0.7201379680285598, 'bagging_freq': 2, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:25:57,802]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:02,460]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:07,268]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:11,252]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:15,570]\u001b[0m Trial 13 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:21,086]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:23,907]\u001b[0m Trial 15 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:27,922]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:33,712]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:37,313]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:42,273]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:46,859]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:50,342]\u001b[0m Trial 21 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:55,710]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:26:59,948]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:03,848]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:08,309]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:13,458]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:16,591]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:21,016]\u001b[0m Trial 28 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:24,997]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:32,574]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 19.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:37,797]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:27:43,562]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:28:21,369]\u001b[0m Trial 33 finished with value: 0.9396167600543115 and parameters: {'lambda_l1': 1.7053800672111565e-08, 'lambda_l2': 0.009838404261194042, 'num_leaves': 205, 'feature_fraction': 0.8042641806396126, 'bagging_fraction': 0.4068986434462453, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:28:28,805]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:28:32,440]\u001b[0m Trial 35 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:29:16,242]\u001b[0m Trial 36 finished with value: 0.9376948232009332 and parameters: {'lambda_l1': 0.00025741391284768883, 'lambda_l2': 1.3903821165130423e-05, 'num_leaves': 179, 'feature_fraction': 0.7513521003294499, 'bagging_fraction': 0.6877869004228008, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:29:22,290]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:29:27,666]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:29:33,842]\u001b[0m Trial 39 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:29:38,984]\u001b[0m Trial 40 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:30:13,915]\u001b[0m Trial 41 finished with value: 0.9370828632078178 and parameters: {'lambda_l1': 0.0003772109478239288, 'lambda_l2': 1.4609599162418084e-05, 'num_leaves': 177, 'feature_fraction': 0.7423326328000164, 'bagging_fraction': 0.6856961600540766, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:30:48,039]\u001b[0m Trial 42 finished with value: 0.9382972213191563 and parameters: {'lambda_l1': 0.0005140907001059531, 'lambda_l2': 2.4931963200658e-05, 'num_leaves': 191, 'feature_fraction': 0.7220599193293482, 'bagging_fraction': 0.6449224331658816, 'bagging_freq': 1, 'min_child_samples': 65}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:30:52,499]\u001b[0m Trial 43 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:31:27,381]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 93.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:31:32,050]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:04,026]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 94.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:08,636]\u001b[0m Trial 47 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:14,065]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:17,136]\u001b[0m Trial 49 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:22,164]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:32:56,694]\u001b[0m Trial 51 finished with value: 0.9398653688015146 and parameters: {'lambda_l1': 8.007598044441421e-05, 'lambda_l2': 1.9006501098879968e-05, 'num_leaves': 193, 'feature_fraction': 0.7474479951356153, 'bagging_fraction': 0.6831044198923434, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 18:33:02,134]\u001b[0m Trial 52 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:25,476]\u001b[0m Trial 53 pruned. Trial was pruned at iteration 75.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:30,367]\u001b[0m Trial 54 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:34,302]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:40,033]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:43,381]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:47,728]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:52,240]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:33:56,752]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:34:02,449]\u001b[0m Trial 61 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:34:07,140]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:34:13,623]\u001b[0m Trial 63 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:34:19,092]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:34:57,096]\u001b[0m Trial 65 finished with value: 0.945717236235681 and parameters: {'lambda_l1': 0.0006215692281330095, 'lambda_l2': 0.0025507405649904265, 'num_leaves': 254, 'feature_fraction': 0.684182512525035, 'bagging_fraction': 0.881844626841578, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:04,313]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:08,810]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:13,882]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:18,314]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:21,402]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:35:26,827]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:06,776]\u001b[0m Trial 72 finished with value: 0.9417586200302155 and parameters: {'lambda_l1': 0.0008708841660641308, 'lambda_l2': 5.4043063920808084e-05, 'num_leaves': 224, 'feature_fraction': 0.7182774647861223, 'bagging_fraction': 0.6574176618253137, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:10,840]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:16,482]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:45,510]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 85.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:50,598]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:36:56,577]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:01,576]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:05,793]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:41,797]\u001b[0m Trial 80 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.08921307105729769, 'lambda_l2': 4.975883016961999e-06, 'num_leaves': 214, 'feature_fraction': 0.8281286566426087, 'bagging_fraction': 0.5953173368434432, 'bagging_freq': 1, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:46,363]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:51,749]\u001b[0m Trial 82 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:37:56,140]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:38:00,762]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:38:06,090]\u001b[0m Trial 85 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:38:10,580]\u001b[0m Trial 86 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:38:48,850]\u001b[0m Trial 87 finished with value: 0.9479260293358321 and parameters: {'lambda_l1': 0.02890836192523439, 'lambda_l2': 4.025111402953753e-05, 'num_leaves': 247, 'feature_fraction': 0.8727646045462183, 'bagging_fraction': 0.7437461253889492, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:38:54,311]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:39:36,775]\u001b[0m Trial 89 finished with value: 0.9436136237593468 and parameters: {'lambda_l1': 2.776285425246586e-05, 'lambda_l2': 3.1180901688950096, 'num_leaves': 235, 'feature_fraction': 0.8371612462873237, 'bagging_fraction': 0.6753094794878766, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:39:41,160]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:15,410]\u001b[0m Trial 91 finished with value: 0.9393968369317856 and parameters: {'lambda_l1': 7.819588207346975e-05, 'lambda_l2': 9.678552881824005e-06, 'num_leaves': 191, 'feature_fraction': 0.7476569605668175, 'bagging_fraction': 0.6665662519078225, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:20,661]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:23,908]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:30,385]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:34,950]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:41,586]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:46,857]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:49,830]\u001b[0m Trial 98 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:40:55,460]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 100\n", "Best trial:\n", " Value: 0.8423533686485246\n", " Params: \n", " lambda_l1: 6.031700696477528e-05\n", " lambda_l2: 0.24394450720600092\n", " num_leaves: 33\n", " feature_fraction: 0.48861458318031864\n", " bagging_fraction: 0.9555014696451426\n", " bagging_freq: 1\n", " min_child_samples: 79\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.023061 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2262\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.347629\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508300\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.444681\n", "[100]\tvalid_0's multi_logloss: 0.37701\n", "[150]\tvalid_0's multi_logloss: 0.33841\n", "[200]\tvalid_0's multi_logloss: 0.311971\n", "[250]\tvalid_0's multi_logloss: 0.291695\n", "[300]\tvalid_0's multi_logloss: 0.274889\n", "[350]\tvalid_0's multi_logloss: 0.261281\n", "[400]\tvalid_0's multi_logloss: 0.250771\n", "[450]\tvalid_0's multi_logloss: 0.240104\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[500]\tvalid_0's multi_logloss: 0.230589\n", "[550]\tvalid_0's multi_logloss: 0.222278\n", "[600]\tvalid_0's multi_logloss: 0.215255\n", "[650]\tvalid_0's multi_logloss: 0.208483\n", "[700]\tvalid_0's multi_logloss: 0.202692\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[750]\tvalid_0's multi_logloss: 0.196167\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[800]\tvalid_0's multi_logloss: 0.190411\n", "[850]\tvalid_0's multi_logloss: 0.185117\n", "[900]\tvalid_0's multi_logloss: 0.18105\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[950]\tvalid_0's multi_logloss: 0.176702\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1000]\tvalid_0's multi_logloss: 0.172928\n", "[1050]\tvalid_0's multi_logloss: 0.169457\n", "[1100]\tvalid_0's multi_logloss: 0.165919\n", "[1150]\tvalid_0's multi_logloss: 0.163006\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1200]\tvalid_0's multi_logloss: 0.15967\n", "[1250]\tvalid_0's multi_logloss: 0.156712\n", "[1300]\tvalid_0's multi_logloss: 0.154223\n", "[1350]\tvalid_0's multi_logloss: 0.151783\n", "[1400]\tvalid_0's multi_logloss: 0.149313\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1450]\tvalid_0's multi_logloss: 0.147362\n", "[1500]\tvalid_0's multi_logloss: 0.145167\n", "[1550]\tvalid_0's multi_logloss: 0.143229\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1600]\tvalid_0's multi_logloss: 0.141306\n", "[1650]\tvalid_0's multi_logloss: 0.139419\n", "[1700]\tvalid_0's multi_logloss: 0.137686\n", "[1750]\tvalid_0's multi_logloss: 0.136191\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1800]\tvalid_0's multi_logloss: 0.134525\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1850]\tvalid_0's multi_logloss: 0.13314\n", "[1900]\tvalid_0's multi_logloss: 0.131585\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1950]\tvalid_0's multi_logloss: 0.130162\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2000]\tvalid_0's multi_logloss: 0.128702\n", "[2050]\tvalid_0's multi_logloss: 0.127237\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2100]\tvalid_0's multi_logloss: 0.125969\n", "[2150]\tvalid_0's multi_logloss: 0.125018\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2200]\tvalid_0's multi_logloss: 0.124019\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2250]\tvalid_0's multi_logloss: 0.122787\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2300]\tvalid_0's multi_logloss: 0.121904\n", "[2350]\tvalid_0's multi_logloss: 0.120813\n", "[2400]\tvalid_0's multi_logloss: 0.119856\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2450]\tvalid_0's multi_logloss: 0.119105\n", "[2500]\tvalid_0's multi_logloss: 0.118181\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2550]\tvalid_0's multi_logloss: 0.117233\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2600]\tvalid_0's multi_logloss: 0.11653\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2650]\tvalid_0's multi_logloss: 0.115597\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2700]\tvalid_0's multi_logloss: 0.114851\n", "[2750]\tvalid_0's multi_logloss: 0.114109\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2800]\tvalid_0's multi_logloss: 0.113475\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2850]\tvalid_0's multi_logloss: 0.112871\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2900]\tvalid_0's multi_logloss: 0.112261\n", "[2950]\tvalid_0's multi_logloss: 0.111637\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3000]\tvalid_0's multi_logloss: 0.111155\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3050]\tvalid_0's multi_logloss: 0.110524\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3100]\tvalid_0's multi_logloss: 0.109987\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3150]\tvalid_0's multi_logloss: 0.1095\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3200]\tvalid_0's multi_logloss: 0.108922\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3250]\tvalid_0's multi_logloss: 0.108389\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3300]\tvalid_0's multi_logloss: 0.107919\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[3350]\tvalid_0's multi_logloss: 0.107463\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3400]\tvalid_0's multi_logloss: 0.107071\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3450]\tvalid_0's multi_logloss: 0.106658\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3500]\tvalid_0's multi_logloss: 0.106334\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3550]\tvalid_0's multi_logloss: 0.105908\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3600]\tvalid_0's multi_logloss: 0.105497\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3650]\tvalid_0's multi_logloss: 0.105035\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3700]\tvalid_0's multi_logloss: 0.104725\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3750]\tvalid_0's multi_logloss: 0.104382\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3800]\tvalid_0's multi_logloss: 0.104083\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3850]\tvalid_0's multi_logloss: 0.103648\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3900]\tvalid_0's multi_logloss: 0.103237\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3950]\tvalid_0's multi_logloss: 0.102936\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4000]\tvalid_0's multi_logloss: 0.102547\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4050]\tvalid_0's multi_logloss: 0.102233\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4100]\tvalid_0's multi_logloss: 0.102018\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4150]\tvalid_0's multi_logloss: 0.10177\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4200]\tvalid_0's multi_logloss: 0.101442\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4250]\tvalid_0's multi_logloss: 0.101199\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4300]\tvalid_0's multi_logloss: 0.101029\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4350]\tvalid_0's multi_logloss: 0.100696\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4400]\tvalid_0's multi_logloss: 0.100538\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4450]\tvalid_0's multi_logloss: 0.100337\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4500]\tvalid_0's multi_logloss: 0.100081\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4550]\tvalid_0's multi_logloss: 0.0998246\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4600]\tvalid_0's multi_logloss: 0.0996299\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4650]\tvalid_0's multi_logloss: 0.0994822\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4700]\tvalid_0's multi_logloss: 0.0992758\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4750]\tvalid_0's multi_logloss: 0.0989493\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4800]\tvalid_0's multi_logloss: 0.098729\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4850]\tvalid_0's multi_logloss: 0.0984868\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4900]\tvalid_0's multi_logloss: 0.0983602\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4950]\tvalid_0's multi_logloss: 0.0982792\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[5000]\tvalid_0's multi_logloss: 0.0981563\n", "Did not meet early stopping. Best iteration is:\n", "[4998]\tvalid_0's multi_logloss: 0.098148\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 18:56:02,686]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:56:09,367]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:56:45,840]\u001b[0m Trial 102 finished with value: 0.9407450612916181 and parameters: {'lambda_l1': 8.197882698131674e-05, 'lambda_l2': 0.00021208640393700757, 'num_leaves': 192, 'feature_fraction': 0.7731692868777431, 'bagging_fraction': 0.6735311455374017, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:56:50,126]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:57:26,832]\u001b[0m Trial 104 finished with value: 0.9414526400336578 and parameters: {'lambda_l1': 1.2795362377766212e-05, 'lambda_l2': 1.8844145747184915e-05, 'num_leaves': 217, 'feature_fraction': 0.7191746432394106, 'bagging_fraction': 0.7098988463324459, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:57:32,559]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:57:37,952]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:57:40,971]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:57:45,702]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:58:22,795]\u001b[0m Trial 109 finished with value: 0.942236713774837 and parameters: {'lambda_l1': 2.209464573982675e-06, 'lambda_l2': 8.287334662719688e-06, 'num_leaves': 207, 'feature_fraction': 0.7665224568657326, 'bagging_fraction': 0.6416358656740727, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:58:27,752]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:58:32,799]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:01,076]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 88.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:06,071]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:11,302]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:15,597]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:19,903]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 18:59:33,007]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 37.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:00:13,094]\u001b[0m Trial 118 finished with value: 0.9443212025013865 and parameters: {'lambda_l1': 0.002942524852825639, 'lambda_l2': 0.011610635505416795, 'num_leaves': 203, 'feature_fraction': 0.9531605764605278, 'bagging_fraction': 0.630189454389572, 'bagging_freq': 4, 'min_child_samples': 42}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:00:17,760]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:00:23,215]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:00:57,349]\u001b[0m Trial 121 finished with value: 0.9405347000439847 and parameters: {'lambda_l1': 7.143416155152381e-05, 'lambda_l2': 0.00017466919367877497, 'num_leaves': 193, 'feature_fraction': 0.7734033889080848, 'bagging_fraction': 0.6713509524709522, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:01:33,315]\u001b[0m Trial 122 finished with value: 0.9424757606471477 and parameters: {'lambda_l1': 3.671159464706789e-05, 'lambda_l2': 3.9461792591530047e-05, 'num_leaves': 209, 'feature_fraction': 0.7582936549212366, 'bagging_fraction': 0.6671570260041155, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:01:39,469]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:15,641]\u001b[0m Trial 124 finished with value: 0.9422271518999445 and parameters: {'lambda_l1': 9.126230817825131e-08, 'lambda_l2': 9.42798578840402e-06, 'num_leaves': 212, 'feature_fraction': 0.7770330244830586, 'bagging_fraction': 0.6480274535778553, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:20,777]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:26,240]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:45,146]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 61.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:50,386]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:53,658]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:02:58,360]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:03:15,101]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 50.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:03:50,148]\u001b[0m Trial 132 finished with value: 0.9417873056548928 and parameters: {'lambda_l1': 8.072026862454635e-05, 'lambda_l2': 8.510873972830464e-05, 'num_leaves': 201, 'feature_fraction': 0.7644190714627204, 'bagging_fraction': 0.679994557223264, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:03:54,573]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:03:59,616]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:03,756]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:07,932]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:14,765]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:19,167]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:54,001]\u001b[0m Trial 139 finished with value: 0.942370580023331 and parameters: {'lambda_l1': 3.232550131286934e-06, 'lambda_l2': 0.0006916472301971968, 'num_leaves': 202, 'feature_fraction': 0.8343548844441779, 'bagging_fraction': 0.8476577552042586, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:04:58,456]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:05:04,372]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:05:41,758]\u001b[0m Trial 142 finished with value: 0.9466256143504619 and parameters: {'lambda_l1': 2.980896851114429e-05, 'lambda_l2': 2.913417349261647e-05, 'num_leaves': 239, 'feature_fraction': 0.763135869473294, 'bagging_fraction': 0.7513284296958156, 'bagging_freq': 1, 'min_child_samples': 62}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:05:46,991]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:05:53,817]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:06:30,154]\u001b[0m Trial 145 finished with value: 0.9437474900078408 and parameters: {'lambda_l1': 7.212460033828516e-05, 'lambda_l2': 4.309094661715439e-06, 'num_leaves': 212, 'feature_fraction': 0.7533850653260141, 'bagging_fraction': 0.8916021652632435, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:06,486]\u001b[0m Trial 146 finished with value: 0.9439291656307969 and parameters: {'lambda_l1': 0.0005892869464862208, 'lambda_l2': 1.1242672082140708e-05, 'num_leaves': 223, 'feature_fraction': 0.7312666460187671, 'bagging_fraction': 0.6736774370139369, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:10,387]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:14,676]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:20,110]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:24,141]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:07:28,653]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:08:04,981]\u001b[0m Trial 152 finished with value: 0.9427626168939206 and parameters: {'lambda_l1': 0.0011892669983091146, 'lambda_l2': 5.820028108882431e-05, 'num_leaves': 224, 'feature_fraction': 0.7351610234838635, 'bagging_fraction': 0.6511917047811185, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 19:08:41,831]\u001b[0m Trial 153 finished with value: 0.9482989424566369 and parameters: {'lambda_l1': 0.00037447863194847096, 'lambda_l2': 2.1542282896260357e-05, 'num_leaves': 234, 'feature_fraction': 0.9144698353599378, 'bagging_fraction': 0.6832488195447302, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:08:47,021]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:08:52,838]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:08:58,447]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:09:02,729]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:09:09,544]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:09:13,126]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:09:40,786]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 88.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:09:56,557]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 42.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:01,280]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:37,093]\u001b[0m Trial 163 finished with value: 0.9425331318965022 and parameters: {'lambda_l1': 6.397677155805411e-08, 'lambda_l2': 0.00039621691928490413, 'num_leaves': 206, 'feature_fraction': 0.7667580883039189, 'bagging_fraction': 0.657903822017336, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:41,857]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:47,583]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:52,424]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:10:58,913]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:11:03,909]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:11:09,381]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:11:13,686]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:11:49,897]\u001b[0m Trial 171 finished with value: 0.9430112256411237 and parameters: {'lambda_l1': 1.102134869211263e-07, 'lambda_l2': 7.503180286760575e-06, 'num_leaves': 214, 'feature_fraction': 0.783389889391717, 'bagging_fraction': 0.6498041172731315, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:12:26,420]\u001b[0m Trial 172 finished with value: 0.9431355300147253 and parameters: {'lambda_l1': 1.0270752816234116e-07, 'lambda_l2': 1.0817935884223942e-05, 'num_leaves': 211, 'feature_fraction': 0.7583412868303638, 'bagging_fraction': 0.6795874395289168, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:13:04,383]\u001b[0m Trial 173 finished with value: 0.9439004800061196 and parameters: {'lambda_l1': 8.279755593895644e-08, 'lambda_l2': 3.6313073067998552e-06, 'num_leaves': 230, 'feature_fraction': 0.7727565832075506, 'bagging_fraction': 0.6452464868388876, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:13:10,975]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:13:46,828]\u001b[0m Trial 175 finished with value: 0.9445315637490199 and parameters: {'lambda_l1': 0.0002353217979236286, 'lambda_l2': 3.171263348326624e-05, 'num_leaves': 218, 'feature_fraction': 0.8203542425794451, 'bagging_fraction': 0.6588367468604717, 'bagging_freq': 1, 'min_child_samples': 76}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:13:51,827]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:13:55,900]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:14:32,835]\u001b[0m Trial 178 finished with value: 0.9489778355739994 and parameters: {'lambda_l1': 1.0560192692698665e-08, 'lambda_l2': 0.0002765136930724561, 'num_leaves': 246, 'feature_fraction': 0.7669065013864841, 'bagging_fraction': 0.6258778389490884, 'bagging_freq': 1, 'min_child_samples': 25}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:15:08,531]\u001b[0m Trial 179 finished with value: 0.9418638006540322 and parameters: {'lambda_l1': 5.8099076285804865e-08, 'lambda_l2': 9.604379213115968e-06, 'num_leaves': 208, 'feature_fraction': 0.7900315064460727, 'bagging_fraction': 0.6778057274702293, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:15:14,384]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:15:22,172]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 21.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:15:32,935]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 32.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:16:09,350]\u001b[0m Trial 183 finished with value: 0.9439865368801514 and parameters: {'lambda_l1': 7.039591924287473e-08, 'lambda_l2': 2.5146082118354423e-05, 'num_leaves': 221, 'feature_fraction': 0.7397390119318659, 'bagging_fraction': 0.672737210757752, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:16:13,980]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:16:18,553]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:16:55,502]\u001b[0m Trial 186 finished with value: 0.9441682125031076 and parameters: {'lambda_l1': 1.9294698699801913e-07, 'lambda_l2': 2.5570868408037523e-06, 'num_leaves': 216, 'feature_fraction': 0.7829248911740091, 'bagging_fraction': 0.9993734316792878, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:01,011]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:04,368]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:09,352]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:15,404]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:20,157]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:17:56,849]\u001b[0m Trial 192 finished with value: 0.9449427243693943 and parameters: {'lambda_l1': 3.0670346509523553e-06, 'lambda_l2': 7.037461956773899e-06, 'num_leaves': 226, 'feature_fraction': 0.7927923736502508, 'bagging_fraction': 0.6369345538445389, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:18:02,512]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:18:06,965]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:18:42,532]\u001b[0m Trial 195 finished with value: 0.941882924403817 and parameters: {'lambda_l1': 5.448535054846655e-05, 'lambda_l2': 0.008712731969886423, 'num_leaves': 211, 'feature_fraction': 0.7730826357400354, 'bagging_fraction': 0.6634899720026096, 'bagging_freq': 1, 'min_child_samples': 90}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:19:18,834]\u001b[0m Trial 196 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 6.64569708412817e-05, 'lambda_l2': 0.0027368345553249167, 'num_leaves': 212, 'feature_fraction': 0.8161286103449696, 'bagging_fraction': 0.6899947578536335, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:19:24,121]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:19:28,683]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:19:33,772]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 200\n", "Best trial:\n", " Value: 0.8423533686485246\n", " Params: \n", " lambda_l1: 6.031700696477528e-05\n", " lambda_l2: 0.24394450720600092\n", " num_leaves: 33\n", " feature_fraction: 0.48861458318031864\n", " bagging_fraction: 0.9555014696451426\n", " bagging_freq: 1\n", " min_child_samples: 79\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027211 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2267\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.347629\n", "[LightGBM] [Info] Start training from score -4.112309\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.442887\n", "[100]\tvalid_0's multi_logloss: 0.373911\n", "[150]\tvalid_0's multi_logloss: 0.336668\n", "[200]\tvalid_0's multi_logloss: 0.312308\n", "[250]\tvalid_0's multi_logloss: 0.293877\n", "[300]\tvalid_0's multi_logloss: 0.277526\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[350]\tvalid_0's multi_logloss: 0.262446\n", "[400]\tvalid_0's multi_logloss: 0.251811\n", "[450]\tvalid_0's multi_logloss: 0.240068\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[500]\tvalid_0's multi_logloss: 0.230934\n", "[550]\tvalid_0's multi_logloss: 0.222786\n", "[600]\tvalid_0's multi_logloss: 0.214897\n", "[650]\tvalid_0's multi_logloss: 0.207044\n", "[700]\tvalid_0's multi_logloss: 0.200602\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[750]\tvalid_0's multi_logloss: 0.194382\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[800]\tvalid_0's multi_logloss: 0.189488\n", "[850]\tvalid_0's multi_logloss: 0.185057\n", "[900]\tvalid_0's multi_logloss: 0.180852\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[950]\tvalid_0's multi_logloss: 0.17619\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1000]\tvalid_0's multi_logloss: 0.172508\n", "[1050]\tvalid_0's multi_logloss: 0.169138\n", "[1100]\tvalid_0's multi_logloss: 0.165975\n", "[1150]\tvalid_0's multi_logloss: 0.162635\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1200]\tvalid_0's multi_logloss: 0.15997\n", "[1250]\tvalid_0's multi_logloss: 0.156712\n", "[1300]\tvalid_0's multi_logloss: 0.1538\n", "[1350]\tvalid_0's multi_logloss: 0.151426\n", "[1400]\tvalid_0's multi_logloss: 0.149565\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1450]\tvalid_0's multi_logloss: 0.147572\n", "[1500]\tvalid_0's multi_logloss: 0.14554\n", "[1550]\tvalid_0's multi_logloss: 0.143424\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1600]\tvalid_0's multi_logloss: 0.141495\n", "[1650]\tvalid_0's multi_logloss: 0.139469\n", "[1700]\tvalid_0's multi_logloss: 0.137832\n", "[1750]\tvalid_0's multi_logloss: 0.13608\n", "[1800]\tvalid_0's multi_logloss: 0.134508\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1850]\tvalid_0's multi_logloss: 0.132867\n", "[1900]\tvalid_0's multi_logloss: 0.131194\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1950]\tvalid_0's multi_logloss: 0.1297\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2000]\tvalid_0's multi_logloss: 0.128321\n", "[2050]\tvalid_0's multi_logloss: 0.127052\n", "[2100]\tvalid_0's multi_logloss: 0.125839\n", "[2150]\tvalid_0's multi_logloss: 0.124789\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2200]\tvalid_0's multi_logloss: 0.123598\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2250]\tvalid_0's multi_logloss: 0.122598\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2300]\tvalid_0's multi_logloss: 0.121569\n", "[2350]\tvalid_0's multi_logloss: 0.120579\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2400]\tvalid_0's multi_logloss: 0.119715\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2450]\tvalid_0's multi_logloss: 0.118919\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2500]\tvalid_0's multi_logloss: 0.118045\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2550]\tvalid_0's multi_logloss: 0.117234\n", "[2600]\tvalid_0's multi_logloss: 0.116567\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2650]\tvalid_0's multi_logloss: 0.115834\n", "[2700]\tvalid_0's multi_logloss: 0.115171\n", "[2750]\tvalid_0's multi_logloss: 0.114451\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2800]\tvalid_0's multi_logloss: 0.11376\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2850]\tvalid_0's multi_logloss: 0.113166\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2900]\tvalid_0's multi_logloss: 0.112467\n", "[2950]\tvalid_0's multi_logloss: 0.111564\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3000]\tvalid_0's multi_logloss: 0.111056\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3050]\tvalid_0's multi_logloss: 0.110477\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3100]\tvalid_0's multi_logloss: 0.109703\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3150]\tvalid_0's multi_logloss: 0.109081\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3200]\tvalid_0's multi_logloss: 0.108637\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3250]\tvalid_0's multi_logloss: 0.108172\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3300]\tvalid_0's multi_logloss: 0.107737\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3350]\tvalid_0's multi_logloss: 0.107314\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3400]\tvalid_0's multi_logloss: 0.106692\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3450]\tvalid_0's multi_logloss: 0.106282\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3500]\tvalid_0's multi_logloss: 0.105917\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3550]\tvalid_0's multi_logloss: 0.105631\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3600]\tvalid_0's multi_logloss: 0.105321\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3650]\tvalid_0's multi_logloss: 0.104916\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3700]\tvalid_0's multi_logloss: 0.104489\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3750]\tvalid_0's multi_logloss: 0.104121\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3800]\tvalid_0's multi_logloss: 0.103784\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3850]\tvalid_0's multi_logloss: 0.103477\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3900]\tvalid_0's multi_logloss: 0.103191\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3950]\tvalid_0's multi_logloss: 0.102865\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4000]\tvalid_0's multi_logloss: 0.102531\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4050]\tvalid_0's multi_logloss: 0.102279\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4100]\tvalid_0's multi_logloss: 0.102028\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4150]\tvalid_0's multi_logloss: 0.101774\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4200]\tvalid_0's multi_logloss: 0.101443\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4250]\tvalid_0's multi_logloss: 0.101177\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4300]\tvalid_0's multi_logloss: 0.101004\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4350]\tvalid_0's multi_logloss: 0.100807\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4400]\tvalid_0's multi_logloss: 0.100584\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4450]\tvalid_0's multi_logloss: 0.100384\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4500]\tvalid_0's multi_logloss: 0.100211\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4550]\tvalid_0's multi_logloss: 0.0999913\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4600]\tvalid_0's multi_logloss: 0.0998812\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4650]\tvalid_0's multi_logloss: 0.0997132\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4700]\tvalid_0's multi_logloss: 0.0994868\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4750]\tvalid_0's multi_logloss: 0.0992342\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4800]\tvalid_0's multi_logloss: 0.0989926\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4850]\tvalid_0's multi_logloss: 0.0987583\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "Early stopping, best iteration is:\n", "[4860]\tvalid_0's multi_logloss: 0.0987031\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 19:35:44,152]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:35:51,579]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:35:57,380]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:03,314]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:07,983]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:13,613]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:21,694]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 21.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:25,647]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:30,111]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:37,233]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:42,081]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:49,029]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:36:53,527]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:00,135]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:04,820]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:09,463]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:13,289]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:16,788]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:23,053]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:37:59,676]\u001b[0m Trial 219 finished with value: 0.9434319481363905 and parameters: {'lambda_l1': 1.1871087780899598e-07, 'lambda_l2': 0.00023968457905386466, 'num_leaves': 219, 'feature_fraction': 0.838039247899186, 'bagging_fraction': 0.6772774131242125, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:38:04,526]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:38:41,652]\u001b[0m Trial 221 finished with value: 0.9425618175211795 and parameters: {'lambda_l1': 8.509891041969528e-05, 'lambda_l2': 3.581606389921117e-05, 'num_leaves': 209, 'feature_fraction': 0.8843923083599923, 'bagging_fraction': 0.6707209079942898, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:38:50,823]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 22.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:38:55,262]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:00,904]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:06,074]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:10,767]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:15,341]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:19,652]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:39:57,821]\u001b[0m Trial 229 finished with value: 0.9456311793616492 and parameters: {'lambda_l1': 0.000357294778794299, 'lambda_l2': 4.383896339017896e-05, 'num_leaves': 218, 'feature_fraction': 0.9990787264144749, 'bagging_fraction': 0.6859577686240965, 'bagging_freq': 1, 'min_child_samples': 96}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:03,271]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:07,882]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:12,972]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:21,168]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 22.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:26,938]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:30,734]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:35,341]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:41,601]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:46,145]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:51,441]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:40:56,059]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:01,106]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:06,655]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:11,262]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:48,687]\u001b[0m Trial 244 finished with value: 0.9444550687498805 and parameters: {'lambda_l1': 0.0002036623168757251, 'lambda_l2': 0.006006535254678557, 'num_leaves': 215, 'feature_fraction': 0.9253229899202093, 'bagging_fraction': 0.6880698265447902, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:54,485]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:41:58,496]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:05,675]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:16,623]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 27.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:21,269]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:26,960]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:33,177]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:39,276]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:43,691]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:48,271]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:54,532]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:42:58,714]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:05,669]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:09,875]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:15,085]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:20,144]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:23,667]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:30,759]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:34,913]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:40,565]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:47,697]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 17.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:53,115]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:43:57,659]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:02,170]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:07,650]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:12,015]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:21,808]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 25.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:26,653]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:32,936]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:36,245]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:41,807]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:46,993]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 19:44:52,300]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:44:57,150]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:45:34,096]\u001b[0m Trial 279 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.00031112512112930536, 'lambda_l2': 0.0001880477331732333, 'num_leaves': 220, 'feature_fraction': 0.852594454408188, 'bagging_fraction': 0.6470457333056485, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:45:38,422]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:15,718]\u001b[0m Trial 281 finished with value: 0.9430685968904783 and parameters: {'lambda_l1': 4.45254298816343e-08, 'lambda_l2': 0.001188876991401564, 'num_leaves': 227, 'feature_fraction': 0.7761533930785017, 'bagging_fraction': 0.6795762898644894, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:21,752]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:26,097]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:31,317]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:36,020]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:41,005]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:48,527]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:54,368]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:46:59,084]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:03,641]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:09,582]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:13,893]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:33,116]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 57.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:36,632]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:41,787]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:46,895]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:49,470]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:54,857]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 19:47:58,624]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 300\n", "Best trial:\n", " Value: 0.8423533686485246\n", " Params: \n", " lambda_l1: 6.031700696477528e-05\n", " lambda_l2: 0.24394450720600092\n", " num_leaves: 33\n", " feature_fraction: 0.48861458318031864\n", " bagging_fraction: 0.9555014696451426\n", " bagging_freq: 1\n", " min_child_samples: 79\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.026580 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2258\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n", "[LightGBM] [Info] Start training from score -1.009534\n", "[LightGBM] [Info] Start training from score -0.717940\n", "[LightGBM] [Info] Start training from score -2.789075\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343939\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.445843\n", "[100]\tvalid_0's multi_logloss: 0.377834\n", "[150]\tvalid_0's multi_logloss: 0.341664\n", "[200]\tvalid_0's multi_logloss: 0.316826\n", "[250]\tvalid_0's multi_logloss: 0.296451\n", "[300]\tvalid_0's multi_logloss: 0.281169\n", "[350]\tvalid_0's multi_logloss: 0.265537\n", "[400]\tvalid_0's multi_logloss: 0.254785\n", "[450]\tvalid_0's multi_logloss: 0.243631\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[500]\tvalid_0's multi_logloss: 0.234101\n", "[550]\tvalid_0's multi_logloss: 0.226309\n", "[600]\tvalid_0's multi_logloss: 0.21796\n", "[650]\tvalid_0's multi_logloss: 0.211559\n", "[700]\tvalid_0's multi_logloss: 0.204727\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[750]\tvalid_0's multi_logloss: 0.198816\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[800]\tvalid_0's multi_logloss: 0.193023\n", "[850]\tvalid_0's multi_logloss: 0.188199\n", "[900]\tvalid_0's multi_logloss: 0.184119\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[950]\tvalid_0's multi_logloss: 0.180033\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1000]\tvalid_0's multi_logloss: 0.176498\n", "[1050]\tvalid_0's multi_logloss: 0.172741\n", "[1100]\tvalid_0's multi_logloss: 0.169305\n", "[1150]\tvalid_0's multi_logloss: 0.166002\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1200]\tvalid_0's multi_logloss: 0.162865\n", "[1250]\tvalid_0's multi_logloss: 0.159812\n", "[1300]\tvalid_0's multi_logloss: 0.157137\n", "[1350]\tvalid_0's multi_logloss: 0.154434\n", "[1400]\tvalid_0's multi_logloss: 0.151964\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1450]\tvalid_0's multi_logloss: 0.149897\n", "[1500]\tvalid_0's multi_logloss: 0.14801\n", "[1550]\tvalid_0's multi_logloss: 0.146389\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1600]\tvalid_0's multi_logloss: 0.144423\n", "[1650]\tvalid_0's multi_logloss: 0.142452\n", "[1700]\tvalid_0's multi_logloss: 0.140879\n", "[1750]\tvalid_0's multi_logloss: 0.138966\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1800]\tvalid_0's multi_logloss: 0.137507\n", "[1850]\tvalid_0's multi_logloss: 0.135709\n", "[1900]\tvalid_0's multi_logloss: 0.134087\n", "[1950]\tvalid_0's multi_logloss: 0.13268\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2000]\tvalid_0's multi_logloss: 0.13141\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2050]\tvalid_0's multi_logloss: 0.130194\n", "[2100]\tvalid_0's multi_logloss: 0.129083\n", "[2150]\tvalid_0's multi_logloss: 0.127913\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2200]\tvalid_0's multi_logloss: 0.126733\n", "[2250]\tvalid_0's multi_logloss: 0.125701\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2300]\tvalid_0's multi_logloss: 0.124613\n", "[2350]\tvalid_0's multi_logloss: 0.12328\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2400]\tvalid_0's multi_logloss: 0.122431\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2450]\tvalid_0's multi_logloss: 0.121605\n", "[2500]\tvalid_0's multi_logloss: 0.120838\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2550]\tvalid_0's multi_logloss: 0.119876\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2600]\tvalid_0's multi_logloss: 0.119193\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2650]\tvalid_0's multi_logloss: 0.118348\n", "[2700]\tvalid_0's multi_logloss: 0.117563\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2750]\tvalid_0's multi_logloss: 0.116756\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2800]\tvalid_0's multi_logloss: 0.116241\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2850]\tvalid_0's multi_logloss: 0.11561\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2900]\tvalid_0's multi_logloss: 0.115103\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2950]\tvalid_0's multi_logloss: 0.114529\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3000]\tvalid_0's multi_logloss: 0.113988\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3050]\tvalid_0's multi_logloss: 0.113383\n", "[3100]\tvalid_0's multi_logloss: 0.112689\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3150]\tvalid_0's multi_logloss: 0.112138\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3200]\tvalid_0's multi_logloss: 0.111635\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3250]\tvalid_0's multi_logloss: 0.111011\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3300]\tvalid_0's multi_logloss: 0.110546\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[3350]\tvalid_0's multi_logloss: 0.110069\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3400]\tvalid_0's multi_logloss: 0.109636\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3450]\tvalid_0's multi_logloss: 0.109215\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3500]\tvalid_0's multi_logloss: 0.108815\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3550]\tvalid_0's multi_logloss: 0.108466\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3600]\tvalid_0's multi_logloss: 0.10805\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3650]\tvalid_0's multi_logloss: 0.107592\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3700]\tvalid_0's multi_logloss: 0.107273\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3750]\tvalid_0's multi_logloss: 0.106966\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3800]\tvalid_0's multi_logloss: 0.106652\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3850]\tvalid_0's multi_logloss: 0.106292\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3900]\tvalid_0's multi_logloss: 0.105884\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3950]\tvalid_0's multi_logloss: 0.105564\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4000]\tvalid_0's multi_logloss: 0.105233\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4050]\tvalid_0's multi_logloss: 0.104994\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4100]\tvalid_0's multi_logloss: 0.104723\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4150]\tvalid_0's multi_logloss: 0.104496\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4200]\tvalid_0's multi_logloss: 0.104218\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4250]\tvalid_0's multi_logloss: 0.103959\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4300]\tvalid_0's multi_logloss: 0.103701\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4350]\tvalid_0's multi_logloss: 0.103406\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4400]\tvalid_0's multi_logloss: 0.103227\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4450]\tvalid_0's multi_logloss: 0.102986\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4500]\tvalid_0's multi_logloss: 0.102788\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4550]\tvalid_0's multi_logloss: 0.102466\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4600]\tvalid_0's multi_logloss: 0.102287\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4650]\tvalid_0's multi_logloss: 0.102138\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4700]\tvalid_0's multi_logloss: 0.101951\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4750]\tvalid_0's multi_logloss: 0.101735\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4800]\tvalid_0's multi_logloss: 0.101611\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4850]\tvalid_0's multi_logloss: 0.101362\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4900]\tvalid_0's multi_logloss: 0.101221\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4950]\tvalid_0's multi_logloss: 0.101075\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[5000]\tvalid_0's multi_logloss: 0.10093\n", "Did not meet early stopping. Best iteration is:\n", "[4988]\tvalid_0's multi_logloss: 0.100912\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 20:03:19,270]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:24,024]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:29,003]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:33,769]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:39,428]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:43,005]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:49,279]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:53,868]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:03:59,420]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:04:04,308]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:04:41,514]\u001b[0m Trial 310 finished with value: 0.9448184199957927 and parameters: {'lambda_l1': 6.33710851183545e-08, 'lambda_l2': 0.00011828921524358206, 'num_leaves': 221, 'feature_fraction': 0.7490482980682972, 'bagging_fraction': 0.7149335066227929, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:04:47,751]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:04:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:03,194]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 21.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:08,173]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:13,677]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:18,274]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:24,488]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:29,197]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:33,932]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:39,317]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:43,765]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:50,484]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:05:54,773]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:06:03,444]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 18.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:06:32,442]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 95.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:06:37,822]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:06:42,014]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:21,450]\u001b[0m Trial 328 finished with value: 0.9455738081122946 and parameters: {'lambda_l1': 3.217868638779184e-05, 'lambda_l2': 5.750590102813345e-05, 'num_leaves': 226, 'feature_fraction': 0.7826114441623828, 'bagging_fraction': 0.6679740332158755, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:27,179]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:32,088]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:37,170]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:42,279]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:46,023]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:53,069]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:07:57,424]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:25,834]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 78.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:30,146]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:35,020]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:41,479]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:45,294]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:51,071]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 12.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:08:56,090]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:01,151]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:05,433]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:42,571]\u001b[0m Trial 345 finished with value: 0.9442255837524621 and parameters: {'lambda_l1': 1.4659439917902066e-06, 'lambda_l2': 9.096938742062834e-06, 'num_leaves': 223, 'feature_fraction': 0.78931856475032, 'bagging_fraction': 0.687754997409419, 'bagging_freq': 1, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:47,319]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:52,425]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:09:57,385]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:01,115]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:07,462]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:11,891]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:17,813]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:22,777]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:28,182]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:34,019]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:38,990]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:49,735]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 31.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:55,530]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:10:59,834]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:11:04,538]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:11:09,697]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:11:47,302]\u001b[0m Trial 362 finished with value: 0.9451052762425657 and parameters: {'lambda_l1': 0.17345143397850926, 'lambda_l2': 1.530418515864943e-05, 'num_leaves': 227, 'feature_fraction': 0.7794541612455614, 'bagging_fraction': 0.8921910315517494, 'bagging_freq': 1, 'min_child_samples': 71}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:11:52,146]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:11:56,135]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:00,790]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:05,348]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:10,259]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:14,597]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:21,394]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:25,610]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:31,248]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:35,702]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:41,410]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:49,192]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 22.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:54,613]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:12:59,016]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 20:13:20,403]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 65.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:13:25,274]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:13:31,727]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:13:36,235]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:13,363]\u001b[0m Trial 381 finished with value: 0.9444646306247729 and parameters: {'lambda_l1': 0.00013491606868735477, 'lambda_l2': 2.4016424269924617e-06, 'num_leaves': 224, 'feature_fraction': 0.7645040960635633, 'bagging_fraction': 0.6776788045685024, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:21,686]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 17.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:26,224]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:30,257]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:35,262]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:40,837]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:14:46,456]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:23,408]\u001b[0m Trial 388 finished with value: 0.9461570824807328 and parameters: {'lambda_l1': 0.03582500455746271, 'lambda_l2': 1.361737608270606e-06, 'num_leaves': 236, 'feature_fraction': 0.7947192105026746, 'bagging_fraction': 0.7121032476222902, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:29,081]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:35,152]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:40,204]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:45,684]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:49,921]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:15:54,703]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:16:00,737]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:16:05,612]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:16:09,888]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:16:14,330]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:16:19,744]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 400\n", "Best trial:\n", " Value: 0.8423533686485246\n", " Params: \n", " lambda_l1: 6.031700696477528e-05\n", " lambda_l2: 0.24394450720600092\n", " num_leaves: 33\n", " feature_fraction: 0.48861458318031864\n", " bagging_fraction: 0.9555014696451426\n", " bagging_freq: 1\n", " min_child_samples: 79\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027687 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2260\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n", "[LightGBM] [Info] Start training from score -1.009528\n", "[LightGBM] [Info] Start training from score -0.717945\n", "[LightGBM] [Info] Start training from score -2.789114\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343871\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.44539\n", "[100]\tvalid_0's multi_logloss: 0.378913\n", "[150]\tvalid_0's multi_logloss: 0.340888\n", "[200]\tvalid_0's multi_logloss: 0.31566\n", "[250]\tvalid_0's multi_logloss: 0.29524\n", "[300]\tvalid_0's multi_logloss: 0.278682\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[350]\tvalid_0's multi_logloss: 0.263956\n", "[400]\tvalid_0's multi_logloss: 0.252259\n", "[450]\tvalid_0's multi_logloss: 0.241152\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[500]\tvalid_0's multi_logloss: 0.231783\n", "[550]\tvalid_0's multi_logloss: 0.223767\n", "[600]\tvalid_0's multi_logloss: 0.215283\n", "[650]\tvalid_0's multi_logloss: 0.207762\n", "[700]\tvalid_0's multi_logloss: 0.201304\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[750]\tvalid_0's multi_logloss: 0.19581\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[800]\tvalid_0's multi_logloss: 0.190409\n", "[850]\tvalid_0's multi_logloss: 0.185526\n", "[900]\tvalid_0's multi_logloss: 0.18142\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[950]\tvalid_0's multi_logloss: 0.177042\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1000]\tvalid_0's multi_logloss: 0.173195\n", "[1050]\tvalid_0's multi_logloss: 0.168927\n", "[1100]\tvalid_0's multi_logloss: 0.165216\n", "[1150]\tvalid_0's multi_logloss: 0.16232\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1200]\tvalid_0's multi_logloss: 0.159091\n", "[1250]\tvalid_0's multi_logloss: 0.156049\n", "[1300]\tvalid_0's multi_logloss: 0.153303\n", "[1350]\tvalid_0's multi_logloss: 0.150912\n", "[1400]\tvalid_0's multi_logloss: 0.148654\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1450]\tvalid_0's multi_logloss: 0.1463\n", "[1500]\tvalid_0's multi_logloss: 0.144477\n", "[1550]\tvalid_0's multi_logloss: 0.142652\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1600]\tvalid_0's multi_logloss: 0.140806\n", "[1650]\tvalid_0's multi_logloss: 0.138759\n", "[1700]\tvalid_0's multi_logloss: 0.137062\n", "[1750]\tvalid_0's multi_logloss: 0.13547\n", "[1800]\tvalid_0's multi_logloss: 0.134167\n", "[1850]\tvalid_0's multi_logloss: 0.132412\n", "[1900]\tvalid_0's multi_logloss: 0.13077\n", "[1950]\tvalid_0's multi_logloss: 0.129542\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2000]\tvalid_0's multi_logloss: 0.128147\n", "[2050]\tvalid_0's multi_logloss: 0.126764\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2100]\tvalid_0's multi_logloss: 0.125437\n", "[2150]\tvalid_0's multi_logloss: 0.124289\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2200]\tvalid_0's multi_logloss: 0.123232\n", "[2250]\tvalid_0's multi_logloss: 0.122371\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2300]\tvalid_0's multi_logloss: 0.121494\n", "[2350]\tvalid_0's multi_logloss: 0.120514\n", "[2400]\tvalid_0's multi_logloss: 0.119568\n", "[2450]\tvalid_0's multi_logloss: 0.118828\n", "[2500]\tvalid_0's multi_logloss: 0.117923\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2550]\tvalid_0's multi_logloss: 0.116858\n", "[2600]\tvalid_0's multi_logloss: 0.116208\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2650]\tvalid_0's multi_logloss: 0.115557\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2700]\tvalid_0's multi_logloss: 0.114849\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2750]\tvalid_0's multi_logloss: 0.11405\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2800]\tvalid_0's multi_logloss: 0.113411\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2850]\tvalid_0's multi_logloss: 0.112686\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2900]\tvalid_0's multi_logloss: 0.11198\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2950]\tvalid_0's multi_logloss: 0.111297\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3000]\tvalid_0's multi_logloss: 0.11087\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3050]\tvalid_0's multi_logloss: 0.110211\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3100]\tvalid_0's multi_logloss: 0.109625\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3150]\tvalid_0's multi_logloss: 0.109116\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3200]\tvalid_0's multi_logloss: 0.108663\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3250]\tvalid_0's multi_logloss: 0.108126\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3300]\tvalid_0's multi_logloss: 0.107707\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3350]\tvalid_0's multi_logloss: 0.107195\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3400]\tvalid_0's multi_logloss: 0.106732\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3450]\tvalid_0's multi_logloss: 0.106219\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3500]\tvalid_0's multi_logloss: 0.105908\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3550]\tvalid_0's multi_logloss: 0.105555\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3600]\tvalid_0's multi_logloss: 0.105145\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3650]\tvalid_0's multi_logloss: 0.104808\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3700]\tvalid_0's multi_logloss: 0.104423\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3750]\tvalid_0's multi_logloss: 0.104159\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3800]\tvalid_0's multi_logloss: 0.103877\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3850]\tvalid_0's multi_logloss: 0.103484\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3900]\tvalid_0's multi_logloss: 0.103104\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3950]\tvalid_0's multi_logloss: 0.102836\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4000]\tvalid_0's multi_logloss: 0.102578\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4050]\tvalid_0's multi_logloss: 0.102273\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4100]\tvalid_0's multi_logloss: 0.101997\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4150]\tvalid_0's multi_logloss: 0.101719\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4200]\tvalid_0's multi_logloss: 0.101443\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4250]\tvalid_0's multi_logloss: 0.101108\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4300]\tvalid_0's multi_logloss: 0.100871\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4350]\tvalid_0's multi_logloss: 0.10061\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4400]\tvalid_0's multi_logloss: 0.100397\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4450]\tvalid_0's multi_logloss: 0.100218\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4500]\tvalid_0's multi_logloss: 0.100004\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4550]\tvalid_0's multi_logloss: 0.0997785\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4600]\tvalid_0's multi_logloss: 0.0996502\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4650]\tvalid_0's multi_logloss: 0.0993899\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4700]\tvalid_0's multi_logloss: 0.0991771\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4750]\tvalid_0's multi_logloss: 0.0989407\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4800]\tvalid_0's multi_logloss: 0.0987919\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4850]\tvalid_0's multi_logloss: 0.0985558\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4900]\tvalid_0's multi_logloss: 0.0983888\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4950]\tvalid_0's multi_logloss: 0.098254\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[5000]\tvalid_0's multi_logloss: 0.0981239\n", "Did not meet early stopping. Best iteration is:\n", "[5000]\tvalid_0's multi_logloss: 0.0981239\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 20:31:41,953]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:31:46,431]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:31:50,256]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:31:56,970]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:03,528]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:07,953]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:13,149]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:18,395]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:23,606]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:29,333]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:32:33,493]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:33:10,283]\u001b[0m Trial 411 finished with value: 0.9432215868887571 and parameters: {'lambda_l1': 1.0106430243212002e-07, 'lambda_l2': 1.7338808138171514e-05, 'num_leaves': 211, 'feature_fraction': 0.8525200092071654, 'bagging_fraction': 0.71247365704883, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:33:15,221]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:33:20,036]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:33:32,811]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 35.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:33:36,360]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:13,257]\u001b[0m Trial 416 finished with value: 0.944780172496223 and parameters: {'lambda_l1': 6.61840029683868e-08, 'lambda_l2': 6.189598777012943e-05, 'num_leaves': 229, 'feature_fraction': 0.7543055044592567, 'bagging_fraction': 0.942324205989867, 'bagging_freq': 1, 'min_child_samples': 88}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:18,909]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:23,665]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:29,134]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:33,970]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:39,344]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:44,235]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:47,811]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:53,765]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:34:59,389]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:04,937]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:10,624]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:16,512]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:22,165]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:27,930]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:33,197]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:38,098]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:43,682]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:48,031]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:53,726]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:35:58,420]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:03,775]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:09,687]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:14,370]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:19,980]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:23,347]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:30,224]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:34,705]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:39,922]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:45,883]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:49,825]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:36:56,073]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:00,642]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:05,855]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:10,406]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:15,960]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:22,067]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:26,480]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:31,630]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:36,056]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:40,816]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:46,124]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:50,577]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:37:58,422]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 14.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:02,782]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:09,254]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:13,847]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:19,221]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:23,415]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:28,284]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:33,838]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:38,296]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:43,969]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:48,921]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:54,443]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:38:59,568]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:04,433]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:10,663]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 11.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:14,889]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:20,884]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:25,563]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:30,567]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:36,476]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:39,231]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:44,485]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:49,432]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:39:54,124]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m[I 2021-09-07 20:39:58,219]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:40:02,806]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:40:09,290]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:40:49,570]\u001b[0m Trial 486 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 0.00020397245265135663, 'lambda_l2': 9.093796518313558e-06, 'num_leaves': 214, 'feature_fraction': 0.8663304164529336, 'bagging_fraction': 0.6476610679220174, 'bagging_freq': 4, 'min_child_samples': 87}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:40:54,931]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:00,439]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:05,860]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 13.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:10,637]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:15,141]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:19,440]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:23,700]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:28,019]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:33,694]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:51,434]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 56.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:41:57,024]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:42:02,000]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n", "\u001b[32m[I 2021-09-07 20:42:07,346]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of finished trials: 500\n", "Best trial:\n", " Value: 0.8423533686485246\n", " Params: \n", " lambda_l1: 6.031700696477528e-05\n", " lambda_l2: 0.24394450720600092\n", " num_leaves: 33\n", " feature_fraction: 0.48861458318031864\n", " bagging_fraction: 0.9555014696451426\n", " bagging_freq: 1\n", " min_child_samples: 79\n", "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.025451 seconds.\n", "You can set `force_col_wise=true` to remove the overhead.\n", "[LightGBM] [Info] Total Bins 2268\n", "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n", "[LightGBM] [Info] Start training from score -1.009528\n", "[LightGBM] [Info] Start training from score -0.717945\n", "[LightGBM] [Info] Start training from score -2.789114\n", "[LightGBM] [Info] Start training from score -5.348131\n", "[LightGBM] [Info] Start training from score -4.112163\n", "[LightGBM] [Info] Start training from score -3.508220\n", "[LightGBM] [Info] Start training from score -3.343871\n", "Training until validation scores don't improve for 20 rounds\n", "[50]\tvalid_0's multi_logloss: 0.44195\n", "[100]\tvalid_0's multi_logloss: 0.375935\n", "[150]\tvalid_0's multi_logloss: 0.338811\n", "[200]\tvalid_0's multi_logloss: 0.313199\n", "[250]\tvalid_0's multi_logloss: 0.294071\n", "[300]\tvalid_0's multi_logloss: 0.277107\n", "[350]\tvalid_0's multi_logloss: 0.264836\n", "[400]\tvalid_0's multi_logloss: 0.252349\n", "[450]\tvalid_0's multi_logloss: 0.241412\n", "[500]\tvalid_0's multi_logloss: 0.231972\n", "[550]\tvalid_0's multi_logloss: 0.22405\n", "[600]\tvalid_0's multi_logloss: 0.217065\n", "[650]\tvalid_0's multi_logloss: 0.210183\n", "[700]\tvalid_0's multi_logloss: 0.204259\n", "[750]\tvalid_0's multi_logloss: 0.198327\n", "[800]\tvalid_0's multi_logloss: 0.192336\n", "[850]\tvalid_0's multi_logloss: 0.187595\n", "[900]\tvalid_0's multi_logloss: 0.183324\n", "[950]\tvalid_0's multi_logloss: 0.178581\n", "[1000]\tvalid_0's multi_logloss: 0.174766\n", "[1050]\tvalid_0's multi_logloss: 0.171164\n", "[1100]\tvalid_0's multi_logloss: 0.167795\n", "[1150]\tvalid_0's multi_logloss: 0.164106\n", "[1200]\tvalid_0's multi_logloss: 0.160688\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1250]\tvalid_0's multi_logloss: 0.157639\n", "[1300]\tvalid_0's multi_logloss: 0.155048\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1350]\tvalid_0's multi_logloss: 0.152564\n", "[1400]\tvalid_0's multi_logloss: 0.14986\n", "[1450]\tvalid_0's multi_logloss: 0.147446\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1500]\tvalid_0's multi_logloss: 0.145395\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1550]\tvalid_0's multi_logloss: 0.143255\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1600]\tvalid_0's multi_logloss: 0.141166\n", "[1650]\tvalid_0's multi_logloss: 0.139344\n", "[1700]\tvalid_0's multi_logloss: 0.137731\n", "[1750]\tvalid_0's multi_logloss: 0.136312\n", "[1800]\tvalid_0's multi_logloss: 0.134798\n", "[1850]\tvalid_0's multi_logloss: 0.133435\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[1900]\tvalid_0's multi_logloss: 0.131893\n", "[1950]\tvalid_0's multi_logloss: 0.130715\n", "[2000]\tvalid_0's multi_logloss: 0.129298\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2050]\tvalid_0's multi_logloss: 0.127718\n", "[2100]\tvalid_0's multi_logloss: 0.126323\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2150]\tvalid_0's multi_logloss: 0.125164\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2200]\tvalid_0's multi_logloss: 0.124135\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2250]\tvalid_0's multi_logloss: 0.123227\n", "[2300]\tvalid_0's multi_logloss: 0.122213\n", "[2350]\tvalid_0's multi_logloss: 0.121182\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2400]\tvalid_0's multi_logloss: 0.120412\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2450]\tvalid_0's multi_logloss: 0.119649\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2500]\tvalid_0's multi_logloss: 0.118669\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2550]\tvalid_0's multi_logloss: 0.11796\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2600]\tvalid_0's multi_logloss: 0.11723\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2650]\tvalid_0's multi_logloss: 0.116418\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2700]\tvalid_0's multi_logloss: 0.11571\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2750]\tvalid_0's multi_logloss: 0.114968\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2800]\tvalid_0's multi_logloss: 0.114305\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2850]\tvalid_0's multi_logloss: 0.113767\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[2900]\tvalid_0's multi_logloss: 0.113049\n", "[2950]\tvalid_0's multi_logloss: 0.112492\n", "[3000]\tvalid_0's multi_logloss: 0.111726\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3050]\tvalid_0's multi_logloss: 0.111202\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3100]\tvalid_0's multi_logloss: 0.110604\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3150]\tvalid_0's multi_logloss: 0.110046\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3200]\tvalid_0's multi_logloss: 0.109533\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3250]\tvalid_0's multi_logloss: 0.109078\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3300]\tvalid_0's multi_logloss: 0.108525\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3350]\tvalid_0's multi_logloss: 0.108166\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3400]\tvalid_0's multi_logloss: 0.107835\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3450]\tvalid_0's multi_logloss: 0.107511\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3500]\tvalid_0's multi_logloss: 0.107145\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3550]\tvalid_0's multi_logloss: 0.106771\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3600]\tvalid_0's multi_logloss: 0.106495\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3650]\tvalid_0's multi_logloss: 0.106142\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3700]\tvalid_0's multi_logloss: 0.105776\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3750]\tvalid_0's multi_logloss: 0.105335\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3800]\tvalid_0's multi_logloss: 0.104977\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3850]\tvalid_0's multi_logloss: 0.104609\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3900]\tvalid_0's multi_logloss: 0.104318\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[3950]\tvalid_0's multi_logloss: 0.104044\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4000]\tvalid_0's multi_logloss: 0.10378\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4050]\tvalid_0's multi_logloss: 0.103526\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4100]\tvalid_0's multi_logloss: 0.103203\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4150]\tvalid_0's multi_logloss: 0.10292\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4200]\tvalid_0's multi_logloss: 0.102592\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4250]\tvalid_0's multi_logloss: 0.102358\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4300]\tvalid_0's multi_logloss: 0.102136\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4350]\tvalid_0's multi_logloss: 0.101919\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4400]\tvalid_0's multi_logloss: 0.101602\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4450]\tvalid_0's multi_logloss: 0.101265\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4500]\tvalid_0's multi_logloss: 0.10099\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4550]\tvalid_0's multi_logloss: 0.100813\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4600]\tvalid_0's multi_logloss: 0.100597\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4650]\tvalid_0's multi_logloss: 0.100381\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4700]\tvalid_0's multi_logloss: 0.100161\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4750]\tvalid_0's multi_logloss: 0.100034\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4800]\tvalid_0's multi_logloss: 0.0998763\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4850]\tvalid_0's multi_logloss: 0.0996593\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4900]\tvalid_0's multi_logloss: 0.099492\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[4950]\tvalid_0's multi_logloss: 0.0993946\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n", "[5000]\tvalid_0's multi_logloss: 0.099238\n", "Did not meet early stopping. Best iteration is:\n", "[5000]\tvalid_0's multi_logloss: 0.099238\n", "Wall time: 2h 35min 57s\n" ] } ], "source": [ "%%time\n", "if __name__ == \"__main__\":\n", " test_predict = 0\n", " study = optuna.create_study(\n", " pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\" # 指定是越小越好\n", " )\n", " for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n", " study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n", " train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n", "\n", " print(\"Number of finished trials: {}\".format(len(study.trials)))\n", "\n", " print(\"Best trial:\")\n", " trial = study.best_trial\n", "\n", " print(\" Value: {}\".format(trial.value))\n", "\n", " print(\" Params: \")\n", " for key, value in trial.params.items():\n", " print(\" {}: {}\".format(key, value))\n", " \n", " params = {\"boosting_type\": \"gbdt\",\n", " \"objective\": \"multiclass\",\n", " \"num_class\": 7,\n", " \"metric\": \"multi_logloss\"}\n", " for key, value in trial.params.items():\n", " params[key]=value\n", " \n", " dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n", " dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n", " model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n", " verbose_eval=50,\n", " early_stopping_rounds=20,\n", " num_boost_round=5000)\n", " test_predict += model.predict(test_x)\n", " predict = test_predict/5" ] }, { "cell_type": "code", "execution_count": 73, "id": "d0cad2c6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "使用optuna的多分类分数： 0.968056177067915\n" ] } ], "source": [ "# 评估指标acc，越大越好\n", "print('使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))" ] }, { "cell_type": "markdown", "id": "63f90cf8", "metadata": {}, "source": [ "### 分类任务的结论\n", "不使用optuna的ACC分数是0.8631716636260369，使用的ACC分数是0.968056177067915，提升了0.1048845134418781。\n", "\n", "结合Stacking.ipynb里的单模随机森林最好分数0.9535087055000585甚至Stacking后的融合分数0.9589887988544127进行对比，也都是上涨的。" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" } }, "nbformat": 4, "nbformat_minor": 5 }