Skip to content

Commit 6d73691

Browse files
author
Github Actions
committed
nabenabe0928: Refactoring base dataset splitting functions (#106)
1 parent 4bfb39c commit 6d73691

22 files changed

+452
-404
lines changed

‎refactor_development/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development/_images/sphx_glr_example_custom_configuration_space_thumb.png‎

8 Bytes
Loading

‎refactor_development/_images/sphx_glr_example_resampling_strategy_thumb.png‎

8 Bytes
Loading

‎refactor_development/_images/sphx_glr_example_tabular_classification_thumb.png‎

8 Bytes
Loading

‎refactor_development/_images/sphx_glr_example_tabular_regression_thumb.png‎

8 Bytes
Loading

‎refactor_development/_sources/advanced_tabular/example_custom_configuration_space.rst.txt‎

Lines changed: 25 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7f2ad8808a90> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7fd58f4d3d00> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -80,7 +80,7 @@ the search. Currently, there are two changes that can be made to the space:-
8080
scaler:__choice__, Value: 'StandardScaler'
8181
trainer:StandardTrainer:weighted_loss, Value: True
8282
trainer:__choice__, Value: 'StandardTrainer'
83-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002296924591064453, budget=0), TrajEntry(train_perf=0.14035087719298245, incumbent_id=1, incumbent=Configuration:
83+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001336812973022461, budget=0), TrajEntry(train_perf=0.14619883040935677, incumbent_id=1, incumbent=Configuration:
8484
data_loader:batch_size, Value: 32
8585
encoder:__choice__, Value: 'OneHotEncoder'
8686
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -114,109 +114,19 @@ the search. Currently, there are two changes that can be made to the space:-
114114
scaler:__choice__, Value: 'StandardScaler'
115115
trainer:StandardTrainer:weighted_loss, Value: True
116116
trainer:__choice__, Value: 'StandardTrainer'
117-
, ta_runs=1, ta_time_used=6.5182719230651855, wallclock_time=8.156251907348633, budget=5.555555555555555), TrajEntry(train_perf=0.14035087719298245, incumbent_id=2, incumbent=Configuration:
118-
data_loader:batch_size, Value: 475
119-
encoder:__choice__, Value: 'OneHotEncoder'
120-
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
121-
imputer:categorical_strategy, Value: 'most_frequent'
122-
imputer:numerical_strategy, Value: 'constant_zero'
123-
lr_scheduler:__choice__, Value: 'NoScheduler'
124-
network_backbone:MLPBackbone:activation, Value: 'tanh'
125-
network_backbone:MLPBackbone:dropout_1, Value: 0.579891279191762
126-
network_backbone:MLPBackbone:dropout_2, Value: 0.43202885747368863
127-
network_backbone:MLPBackbone:dropout_3, Value: 0.2053050533304992
128-
network_backbone:MLPBackbone:dropout_4, Value: 0.3628626567848122
129-
network_backbone:MLPBackbone:dropout_5, Value: 0.000687232634536894
130-
network_backbone:MLPBackbone:dropout_6, Value: 0.30779918180581656
131-
network_backbone:MLPBackbone:dropout_7, Value: 0.4566654226669556
132-
network_backbone:MLPBackbone:num_groups, Value: 7
133-
network_backbone:MLPBackbone:num_units_1, Value: 749
134-
network_backbone:MLPBackbone:num_units_2, Value: 751
135-
network_backbone:MLPBackbone:num_units_3, Value: 759
136-
network_backbone:MLPBackbone:num_units_4, Value: 664
137-
network_backbone:MLPBackbone:num_units_5, Value: 219
138-
network_backbone:MLPBackbone:num_units_6, Value: 757
139-
network_backbone:MLPBackbone:num_units_7, Value: 1005
140-
network_backbone:MLPBackbone:use_dropout, Value: True
141-
network_backbone:__choice__, Value: 'MLPBackbone'
142-
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.9640640623783606
143-
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.017233504391813814
144-
network_embedding:LearnedEntityEmbedding:dimension_reduction_2, Value: 0.24122690885917664
145-
network_embedding:LearnedEntityEmbedding:dimension_reduction_3, Value: 0.31247176333246596
146-
network_embedding:LearnedEntityEmbedding:dimension_reduction_4, Value: 0.41504826813841933
147-
network_embedding:LearnedEntityEmbedding:dimension_reduction_5, Value: 0.8395119637200936
148-
network_embedding:LearnedEntityEmbedding:dimension_reduction_6, Value: 0.8208414027523236
149-
network_embedding:LearnedEntityEmbedding:dimension_reduction_7, Value: 0.4284420622613293
150-
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 7
151-
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
152-
network_head:__choice__, Value: 'fully_connected'
153-
network_head:fully_connected:num_layers, Value: 1
154-
network_init:KaimingInit:bias_strategy, Value: 'Zero'
155-
network_init:__choice__, Value: 'KaimingInit'
156-
optimizer:AdamOptimizer:beta1, Value: 0.9770847327434384
157-
optimizer:AdamOptimizer:beta2, Value: 0.9710627513919582
158-
optimizer:AdamOptimizer:lr, Value: 0.00010844892447274338
159-
optimizer:AdamOptimizer:weight_decay, Value: 0.05048412416506887
160-
optimizer:__choice__, Value: 'AdamOptimizer'
161-
scaler:Normalizer:norm, Value: 'max'
162-
scaler:__choice__, Value: 'Normalizer'
163-
trainer:StandardTrainer:weighted_loss, Value: False
164-
trainer:__choice__, Value: 'StandardTrainer'
165-
, ta_runs=11, ta_time_used=135.23650550842285, wallclock_time=163.7278196811676, budget=16.666666666666664), TrajEntry(train_perf=0.13450292397660824, incumbent_id=3, incumbent=Configuration:
166-
data_loader:batch_size, Value: 174
167-
encoder:__choice__, Value: 'OneHotEncoder'
168-
feature_preprocessor:PowerTransformer:standardize, Value: True
169-
feature_preprocessor:__choice__, Value: 'PowerTransformer'
170-
imputer:categorical_strategy, Value: 'most_frequent'
171-
imputer:numerical_strategy, Value: 'most_frequent'
172-
lr_scheduler:ExponentialLR:gamma, Value: 0.9015102188730918
173-
lr_scheduler:__choice__, Value: 'ExponentialLR'
174-
network_backbone:MLPBackbone:activation, Value: 'relu'
175-
network_backbone:MLPBackbone:num_groups, Value: 2
176-
network_backbone:MLPBackbone:num_units_1, Value: 738
177-
network_backbone:MLPBackbone:num_units_2, Value: 371
178-
network_backbone:MLPBackbone:use_dropout, Value: False
179-
network_backbone:__choice__, Value: 'MLPBackbone'
180-
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.5026052624432612
181-
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.7197339745319954
182-
network_embedding:LearnedEntityEmbedding:dimension_reduction_2, Value: 0.9509463920492078
183-
network_embedding:LearnedEntityEmbedding:dimension_reduction_3, Value: 0.5338374300900856
184-
network_embedding:LearnedEntityEmbedding:dimension_reduction_4, Value: 0.2115815132516191
185-
network_embedding:LearnedEntityEmbedding:dimension_reduction_5, Value: 0.3101651859744802
186-
network_embedding:LearnedEntityEmbedding:dimension_reduction_6, Value: 0.724733217572461
187-
network_embedding:LearnedEntityEmbedding:dimension_reduction_7, Value: 0.1459031891772793
188-
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 4
189-
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
190-
network_head:__choice__, Value: 'fully_connected'
191-
network_head:fully_connected:activation, Value: 'tanh'
192-
network_head:fully_connected:num_layers, Value: 3
193-
network_head:fully_connected:units_layer_1, Value: 212
194-
network_head:fully_connected:units_layer_2, Value: 414
195-
network_init:NoInit:bias_strategy, Value: 'Normal'
196-
network_init:__choice__, Value: 'NoInit'
197-
optimizer:RMSpropOptimizer:alpha, Value: 0.5152986504870539
198-
optimizer:RMSpropOptimizer:lr, Value: 0.003121222619830083
199-
optimizer:RMSpropOptimizer:momentum, Value: 0.18725278090773345
200-
optimizer:RMSpropOptimizer:weight_decay, Value: 0.0065085103879551805
201-
optimizer:__choice__, Value: 'RMSpropOptimizer'
202-
scaler:__choice__, Value: 'StandardScaler'
203-
trainer:StandardTrainer:weighted_loss, Value: False
204-
trainer:__choice__, Value: 'StandardTrainer'
205-
, ta_runs=17, ta_time_used=213.5607807636261, wallclock_time=264.5632119178772, budget=50.0)]
206-
{'accuracy': 0.8554913294797688}
117+
, ta_runs=1, ta_time_used=4.047108173370361, wallclock_time=5.4710657596588135, budget=5.555555555555555)]
118+
{'accuracy': 0.8901734104046243}
207119
| | Preprocessing | Estimator | Weight |
208120
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
209-
| 0 | None | RFClassifier | 0.26 |
210-
| 1 | None | ExtraTreesClassifier | 0.24 |
211-
| 2 | SimpleImputer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
212-
| 3 | None | CatBoostClassifier | 0.1 |
213-
| 4 | None | KNNClassifier | 0.1 |
214-
| 5 | SimpleImputer,OneHotEncoder,Normalizer,KernelPCA | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.08 |
215-
| 6 | SimpleImputer,OneHotEncoder,Normalizer,KernelPCA | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
216-
| 7 | SimpleImputer,OneHotEncoder,Normalizer,NoFeaturePreprocessing | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
217-
| 8 | None | LGBMClassifier | 0.02 |
218-
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
219-
<smac.runhistory.runhistory.RunHistory object at 0x7f2ad802a5e0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
121+
| 0 | None | CatBoostClassifier | 0.28 |
122+
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
123+
| 2 | None | ExtraTreesClassifier | 0.18 |
124+
| 3 | SimpleImputer,OneHotEncoder,Normalizer,KernelPCA | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.14 |
125+
| 4 | None | KNNClassifier | 0.06 |
126+
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
127+
| 6 | SimpleImputer,OneHotEncoder,Normalizer,KernelPCA | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
128+
| 7 | None | RFClassifier | 0.04 |
129+
<smac.runhistory.runhistory.RunHistory object at 0x7fd57e5f4d00> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
220130
data_loader:batch_size, Value: 32
221131
encoder:__choice__, Value: 'NoEncoder'
222132
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -248,7 +158,7 @@ the search. Currently, there are two changes that can be made to the space:-
248158
scaler:__choice__, Value: 'StandardScaler'
249159
trainer:StandardTrainer:weighted_loss, Value: True
250160
trainer:__choice__, Value: 'StandardTrainer'
251-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001631021499633789, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
161+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0012526512145996094, budget=0), TrajEntry(train_perf=0.19298245614035092, incumbent_id=1, incumbent=Configuration:
252162
data_loader:batch_size, Value: 32
253163
encoder:__choice__, Value: 'NoEncoder'
254164
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -280,15 +190,17 @@ the search. Currently, there are two changes that can be made to the space:-
280190
scaler:__choice__, Value: 'StandardScaler'
281191
trainer:StandardTrainer:weighted_loss, Value: True
282192
trainer:__choice__, Value: 'StandardTrainer'
283-
, ta_runs=1, ta_time_used=5.196228265762329, wallclock_time=6.9113147258758545, budget=5.555555555555555)]
193+
, ta_runs=1, ta_time_used=3.548128843307495, wallclock_time=4.968382835388184, budget=5.555555555555555)]
284194
{'accuracy': 0.8728323699421965}
285-
| | Preprocessing | Estimator | Weight |
286-
|---:|:----------------|:---------------------|---------:|
287-
| 0 | None | ExtraTreesClassifier | 0.34 |
288-
| 1 | None | CatBoostClassifier | 0.28 |
289-
| 2 | None | RFClassifier | 0.2 |
290-
| 3 | None | KNNClassifier | 0.16 |
291-
| 4 | None | SVC | 0.02 |
195+
| | Preprocessing | Estimator | Weight |
196+
|---:|:-------------------------------------------------|:-------------------------------------------------------------------|---------:|
197+
| 0 | None | CatBoostClassifier | 0.24 |
198+
| 1 | None | RFClassifier | 0.22 |
199+
| 2 | None | ExtraTreesClassifier | 0.18 |
200+
| 3 | None | KNNClassifier | 0.14 |
201+
| 4 | None | LGBMClassifier | 0.1 |
202+
| 5 | SimpleImputer,NoEncoder,MinMaxScaler,KitchenSink | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
203+
| 6 | None | SVC | 0.06 |
292204
293205
294206
@@ -417,7 +329,7 @@ the search. Currently, there are two changes that can be made to the space:-
417329
418330
.. rst-class:: sphx-glr-timing
419331

420-
**Total running time of the script:** ( 11 minutes 46.927 seconds)
332+
**Total running time of the script:** ( 11 minutes 24.080 seconds)
421333

422334

423335
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)