Heart disease detection - Solution#
In this tutorial, we will focus on applying federated learning techniques to a classification problem using Scikit-Learn, a popular machine learning library in Python. We will walk you through the process step by step, from setting up the federated learning environment to evaluating the model’s performance.
Scikit-Learn, also known as sklearn, is a popular machine learning library in Python. It provides a wide range of tools and algorithms for tasks such as data preprocessing, feature selection, model training, and evaluation. Sklearn is widely used for tasks such as classification, regression, clustering, and dimensionality reduction. It offers a user-friendly interface and integrates well with other libraries in the Python ecosystem, making it a go-to choice for many machine learning practitioners and researchers.
%load_ext autoreload
%autoreload 2
Table of content#
The dataset
Task 1: training plan
Task 2: the experment
Task 3: model validation
Tutorial#
The dataset #
The Heart Disease dataset available at https://archive.ics.uci.edu/dataset/45/heart+disease is a widely used dataset in the field of cardiovascular research and machine learning. It contains a collection of medical attributes from patients suspected of having heart disease, along with their corresponding diagnosis (presence or absence of heart disease). The dataset includes information such as age, sex, blood pressure, cholesterol levels, and various other clinical measurements.
It was collected in 4 hospitals in the USA, Switzerland and Hungary. This dataset contains tabular information about 740 patients distributed among these four clients.
A federated version of this dataset has been proposed in Flamby. Following thier actions, we preprocess the dataset by removing missing values and encoding non-binary categorical variables as dummy variables. We finally obtain the following centers:
Number |
Client |
Dataset size |
|---|---|---|
0 |
Cleveland’s Hospital |
303 |
1 |
Hungarian Hospital |
261 |
2 |
Switzerland Hospital |
46 |
3 |
Long Beach Hospital |
130 |
For teaching purposes, we decided to merge: client0 with client3 and client1 with client2. The final federated scenario, in this way, is the following:
client1, with 349 elements
client2, with 391 elements
Task 1: Defining the training plan #
A training plan is a class that defines the four main components of federated model training: the data, the model, he loss and the optimizer. It is responsible for providing custom methods allowing every node to perform the training.
In the case of scikit-learn, Fed-BioMed already does a lot of the heavy lifting for you by providing the FedPerceptron, FedSGDClassifier and FedSGDRegressor classes as training plans. These classes already take care of the model, optimizer, loss function and related dependencies for you, so you only need to define how the data will be loaded.
In this tutorial we are going to use an SGDClassifier, so the related FedSGDClassifier training plan.
Model arguments#
model_args is a dictionary with the arguments related to the model, that will be passed to the Perceptron constructor.
IMPORTANT For classification tasks, you are required to specify the following two fields:
n_features: the number of features in each input sample (in our case, the number of pixels in the images)
n_classes: the number of classes in the target data
Other model arguments depend on the specific model you are using, and are defined in the model definition. Refer to the model documentation
Training arguments#
training_args is a dictionary containing the arguments for the training routine (e.g. batch size, learning rate, epochs, etc.). This will be passed to the routine on the node side.
IMPORTANT To set the training arguments we may either pass them to the Experiment constructor, or set them on an instance with the setter method:
'exp.set_training_arguments(training_args=training_args)'
The setters are available also for single training arguments, like:
'exp.set_aggregator(aggregator=FedAverage)'
TO_DO:
Apply the scaler to your data
Define training args: num_updates, batch_size.
Define model args.
from fedbiomed.common.training_plans import FedSGDRegressor, FedPerceptron, FedSGDClassifier
from fedbiomed.common.data import DataManager
from sklearn.preprocessing import MinMaxScaler
class SkLearnClassifierTrainingPlan(FedSGDClassifier):
def init_dependencies(self):
"""Define additional dependencies.
return ["from torchvision import datasets, transforms",
"from torch.utils.data import DataLoader"]
def training_data(self, batch_size):
In this case, we rely on torchvision functions for preprocessing the images.
"""
return ["from sklearn.preprocessing import MinMaxScaler"]
def training_data(self, batch_size):
df = pd.read_csv(self.dataset_path, delimiter=';')
X = df.iloc[:, :-1]
y = df.iloc[:, -1]
self.scaler = MinMaxScaler()
X = self.scaler.fit_transform(X)
return DataManager(dataset=X, target=y, batch_size=batch_size, shuffle=True)
n_features = 18
n_classes = 2
model_args = { 'max_iter':1000, 'tol': 1e-1 ,
'n_features' : n_features, 'n_classes' : n_classes, 'loss': 'hinge'}
training_args = {
'num_updates': 5,
'batch_size': 128,
'dry_run': False,
}
Task 2: the Experiment #
The experiment enables Federated Learning by orchestrating the training process across multiple nodes. It searches for datasets based on specific tags, uploads the training plan file, sends model and training arguments, tracks and checks training progress, and downloads and aggregates model parameters for the next round.
TO_DO:
Define the tags to select the nodes
Define the used training plan.
Define the number of rounds for the federated process.
TAGS: Replace %%%% in the tags with your username
from fedbiomed.researcher.experiment import Experiment
from fedbiomed.researcher.aggregators.fedavg import FedAverage
tags = ['heart-jupyter-sharkovsky']
rounds = 10
# search for corresponding datasets across nodes datasets
exp = Experiment(tags=tags,
model_args=model_args,
training_plan_class=SkLearnClassifierTrainingPlan,
training_args=training_args,
round_limit=rounds,
aggregator=FedAverage(),
node_selection_strategy=None)
2023-07-03 14:38:00,927 fedbiomed INFO - Messaging researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87 successfully connected to the message broker, object = <fedbiomed.common.messaging.Messaging object at 0x7fce404c4b20>
2023-07-03 14:38:00,959 fedbiomed INFO - Searching dataset with data tags: ['heart-jupyter-sharkovsky'] for all nodes
2023-07-03 14:38:10,973 fedbiomed INFO - Node selected for training -> node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
2023-07-03 14:38:10,974 fedbiomed INFO - Node selected for training -> node_10797f2f-2524-4595-a1c6-f3c67e03add1
2023-07-03 14:38:10,976 fedbiomed INFO - Checking data quality of federated datasets...
2023-07-03 14:38:10,977 fedbiomed DEBUG - Using native Sklearn Optimizer
2023-07-03 14:38:10,978 fedbiomed DEBUG - Model file has been saved: /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/my_model_8b48c219-cb97-4963-b035-4745618fca23.py
2023-07-03 14:38:11,071 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/my_model_8b48c219-cb97-4963-b035-4745618fca23.py successful, with status code 201
2023-07-03 14:38:11,080 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_b7de0161-57ad-4857-ac2b-bd5a43674bb6.mpk successful, with status code 201
2023-07-03 14:38:11,081 fedbiomed INFO - Removing tensorboard logs from previous experiment
# train experiments
exp.run()
2023-07-03 14:39:41,787 fedbiomed INFO - Sampled nodes in round 0 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:39:41,788 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 0, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_b7de0161-57ad-4857-ac2b-bd5a43674bb6.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:39:41,788 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:39:41,789 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 0, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_b7de0161-57ad-4857-ac2b-bd5a43674bb6.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:39:41,790 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:39:42,030 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 1 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.000000
---------
2023-07-03 14:39:42,031 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 1 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.000000
---------
2023-07-03 14:39:42,053 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 1 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 1.006065
---------
2023-07-03 14:39:42,057 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 1 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.852417
---------
2023-07-03 14:39:42,076 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 1 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 3.580541
---------
2023-07-03 14:39:42,078 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 1 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 4.077033
---------
2023-07-03 14:39:42,093 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 1 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 3.068150
---------
2023-07-03 14:39:51,806 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_d42587cb-dd29-460f-887a-910642918d60.mpk
2023-07-03 14:39:51,811 fedbiomed DEBUG - download of file node_params_79d8a21d-99a3-40cf-af83-11ed8868ec9a.mpk successful, with status code 200
2023-07-03 14:39:51,811 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_d84fbfc3-a23c-4195-b5bd-638143b4abac.mpk
2023-07-03 14:39:51,815 fedbiomed DEBUG - download of file node_params_182ef421-d214-40a3-ae8b-d2d4d3e9ceeb.mpk successful, with status code 200
2023-07-03 14:39:51,819 fedbiomed INFO - Nodes that successfully reply in round 0 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:39:51,828 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_c1adb784-b870-4b6d-bf3e-c6cec5578306.mpk successful, with status code 201
2023-07-03 14:39:51,828 fedbiomed INFO - Saved aggregated params for round 0 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_c1adb784-b870-4b6d-bf3e-c6cec5578306.mpk
2023-07-03 14:39:51,829 fedbiomed INFO - Sampled nodes in round 1 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:39:51,829 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 1, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c1adb784-b870-4b6d-bf3e-c6cec5578306.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:39:51,829 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:39:51,830 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 1, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c1adb784-b870-4b6d-bf3e-c6cec5578306.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:39:51,830 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:39:51,864 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 2 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.049416
---------
2023-07-03 14:39:51,865 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 2 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.027009
---------
2023-07-03 14:39:51,899 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 2 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 1.063915
---------
2023-07-03 14:39:51,904 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 2 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.911342
---------
2023-07-03 14:39:51,923 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 2 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 2.797854
---------
2023-07-03 14:39:51,924 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 2 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 0.870021
---------
2023-07-03 14:39:51,939 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 2 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.679158
---------
2023-07-03 14:40:01,843 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_18867bc3-13a0-43ec-9397-ba4d0b5b05b8.mpk
2023-07-03 14:40:01,848 fedbiomed DEBUG - download of file node_params_a6124a9a-5bfb-4549-b27c-cbf72f5e0d6c.mpk successful, with status code 200
2023-07-03 14:40:01,848 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_227d2e9f-8b09-45ba-bf96-b342bb22c144.mpk
2023-07-03 14:40:01,851 fedbiomed DEBUG - download of file node_params_7cba8c4d-bc05-4b41-acd7-f409f5b94d3a.mpk successful, with status code 200
2023-07-03 14:40:01,853 fedbiomed INFO - Nodes that successfully reply in round 1 ['node_10797f2f-2524-4595-a1c6-f3c67e03add1', 'node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1']
2023-07-03 14:40:01,863 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_c66687a0-6b7e-41ce-b9fc-7d05339bdc56.mpk successful, with status code 201
2023-07-03 14:40:01,863 fedbiomed INFO - Saved aggregated params for round 1 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_c66687a0-6b7e-41ce-b9fc-7d05339bdc56.mpk
2023-07-03 14:40:01,864 fedbiomed INFO - Sampled nodes in round 2 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:01,864 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 2, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c66687a0-6b7e-41ce-b9fc-7d05339bdc56.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:01,865 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:01,866 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 2, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c66687a0-6b7e-41ce-b9fc-7d05339bdc56.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:01,867 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:01,899 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 3 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.172746
---------
2023-07-03 14:40:01,942 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 3 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.089842
---------
2023-07-03 14:40:01,943 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 3 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 0.969353
---------
2023-07-03 14:40:01,946 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 3 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 0.823574
---------
2023-07-03 14:40:01,956 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 3 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 5.964549
---------
2023-07-03 14:40:01,965 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 3 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.420680
---------
2023-07-03 14:40:01,985 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 3 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.777982
---------
2023-07-03 14:40:11,876 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_8fbbaf35-845f-4a2f-af90-8ad0a3275401.mpk
2023-07-03 14:40:11,880 fedbiomed DEBUG - download of file node_params_e0f7c6dd-6c13-4eef-9ce4-981e6287212a.mpk successful, with status code 200
2023-07-03 14:40:11,881 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_6f5bfd2e-4e6d-4283-97dc-02b3b543e12e.mpk
2023-07-03 14:40:11,884 fedbiomed DEBUG - download of file node_params_f8b8c152-6cb0-439b-b141-d605b6a499e9.mpk successful, with status code 200
2023-07-03 14:40:11,885 fedbiomed INFO - Nodes that successfully reply in round 2 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:11,894 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_e02fc9ff-20e8-4065-8670-5a67fe6b7914.mpk successful, with status code 201
2023-07-03 14:40:11,894 fedbiomed INFO - Saved aggregated params for round 2 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_e02fc9ff-20e8-4065-8670-5a67fe6b7914.mpk
2023-07-03 14:40:11,895 fedbiomed INFO - Sampled nodes in round 3 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:11,895 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 3, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_e02fc9ff-20e8-4065-8670-5a67fe6b7914.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:11,895 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:11,896 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 3, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_e02fc9ff-20e8-4065-8670-5a67fe6b7914.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:11,897 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:11,938 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 4 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.874855
---------
2023-07-03 14:40:11,939 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 4 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.016869
---------
2023-07-03 14:40:11,982 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 4 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.953992
---------
2023-07-03 14:40:11,983 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 4 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 0.983414
---------
2023-07-03 14:40:11,987 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 4 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 2.187833
---------
2023-07-03 14:40:11,991 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 4 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.484909
---------
2023-07-03 14:40:12,010 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 4 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 2.560063
---------
2023-07-03 14:40:21,909 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_86313eb7-16fe-427e-b5d2-a2dca7d7520b.mpk
2023-07-03 14:40:21,913 fedbiomed DEBUG - download of file node_params_34662ed3-632e-42ef-9c22-fc909e7f7dc1.mpk successful, with status code 200
2023-07-03 14:40:21,913 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_9f2e0e27-43e6-4378-b983-48ccc61ebf53.mpk
2023-07-03 14:40:21,916 fedbiomed DEBUG - download of file node_params_ff111a95-edfe-4efa-91d4-d563f3149d7a.mpk successful, with status code 200
2023-07-03 14:40:21,918 fedbiomed INFO - Nodes that successfully reply in round 3 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:21,926 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_e2f557b0-ba8c-4190-a3fd-0d9a5400ee49.mpk successful, with status code 201
2023-07-03 14:40:21,927 fedbiomed INFO - Saved aggregated params for round 3 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_e2f557b0-ba8c-4190-a3fd-0d9a5400ee49.mpk
2023-07-03 14:40:21,927 fedbiomed INFO - Sampled nodes in round 4 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:21,927 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 4, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_e2f557b0-ba8c-4190-a3fd-0d9a5400ee49.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:21,928 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:21,929 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 4, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_e2f557b0-ba8c-4190-a3fd-0d9a5400ee49.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:21,929 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:21,968 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 5 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.421098
---------
2023-07-03 14:40:21,969 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 5 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.677845
---------
2023-07-03 14:40:22,010 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 5 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.560698
---------
2023-07-03 14:40:22,010 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 5 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 1.570311
---------
2023-07-03 14:40:22,020 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 5 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 2.906239
---------
2023-07-03 14:40:22,023 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 5 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.144794
---------
2023-07-03 14:40:22,042 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 5 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.139720
---------
2023-07-03 14:40:31,941 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_dd9973c0-9fa0-4d01-adc2-f19aadcaa5fd.mpk
2023-07-03 14:40:31,945 fedbiomed DEBUG - download of file node_params_1ce2ab96-2a7e-43e7-9277-1c3bf1680518.mpk successful, with status code 200
2023-07-03 14:40:31,945 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_0f7907e3-e56b-4d9b-b6f7-e034cd6d351b.mpk
2023-07-03 14:40:31,948 fedbiomed DEBUG - download of file node_params_4326d6f7-895f-4cf6-af05-2b45473b6f6f.mpk successful, with status code 200
2023-07-03 14:40:31,950 fedbiomed INFO - Nodes that successfully reply in round 4 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:31,959 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_66b3d1f3-52f8-41be-bccc-be0240743d63.mpk successful, with status code 201
2023-07-03 14:40:31,959 fedbiomed INFO - Saved aggregated params for round 4 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_66b3d1f3-52f8-41be-bccc-be0240743d63.mpk
2023-07-03 14:40:31,960 fedbiomed INFO - Sampled nodes in round 5 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:31,960 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 5, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_66b3d1f3-52f8-41be-bccc-be0240743d63.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:31,960 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:31,961 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 5, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_66b3d1f3-52f8-41be-bccc-be0240743d63.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:31,961 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:32,014 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 6 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.960151
---------
2023-07-03 14:40:32,018 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 6 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.176058
---------
2023-07-03 14:40:32,051 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 6 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 3.156323
---------
2023-07-03 14:40:32,052 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 6 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 0.865018
---------
2023-07-03 14:40:32,053 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 6 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 3.246942
---------
2023-07-03 14:40:32,079 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 6 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.201610
---------
2023-07-03 14:40:32,080 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 6 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.450477
---------
2023-07-03 14:40:41,974 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_5328fe90-7ce9-455e-9cb1-d7018c8424df.mpk
2023-07-03 14:40:41,978 fedbiomed DEBUG - download of file node_params_101ae7c7-06fc-4c5f-8484-c1d8578e8e3d.mpk successful, with status code 200
2023-07-03 14:40:41,978 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_d2c4c801-1ce1-4fee-b9fd-7025d130932b.mpk
2023-07-03 14:40:41,981 fedbiomed DEBUG - download of file node_params_abdfd281-d477-4f70-b4eb-8eda46fcd2ef.mpk successful, with status code 200
2023-07-03 14:40:41,982 fedbiomed INFO - Nodes that successfully reply in round 5 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:41,991 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_08cc9313-21f1-4c35-b919-bcd0ec20a017.mpk successful, with status code 201
2023-07-03 14:40:41,992 fedbiomed INFO - Saved aggregated params for round 5 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_08cc9313-21f1-4c35-b919-bcd0ec20a017.mpk
2023-07-03 14:40:41,992 fedbiomed INFO - Sampled nodes in round 6 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:41,992 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 6, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_08cc9313-21f1-4c35-b919-bcd0ec20a017.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:41,993 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:41,994 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 6, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_08cc9313-21f1-4c35-b919-bcd0ec20a017.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:41,994 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:42,041 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 7 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.332799
---------
2023-07-03 14:40:42,042 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 7 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.287343
---------
2023-07-03 14:40:42,082 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 7 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 0.000000
---------
2023-07-03 14:40:42,082 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 7 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 3.003612
---------
2023-07-03 14:40:42,085 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 7 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 1.350821
---------
2023-07-03 14:40:42,112 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 7 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.574242
---------
2023-07-03 14:40:42,112 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 7 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.177146
---------
2023-07-03 14:40:52,006 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_7c4193d5-0545-440f-ba95-a3ca6f54b567.mpk
2023-07-03 14:40:52,010 fedbiomed DEBUG - download of file node_params_8152e0d4-d70c-4949-a46d-cae5184a58af.mpk successful, with status code 200
2023-07-03 14:40:52,011 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_38b37e72-dec3-4df1-9771-6a1954f6016d.mpk
2023-07-03 14:40:52,013 fedbiomed DEBUG - download of file node_params_e79562dc-2049-4038-bb28-ff96158484eb.mpk successful, with status code 200
2023-07-03 14:40:52,015 fedbiomed INFO - Nodes that successfully reply in round 6 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:52,023 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_9adf34fb-59ca-4107-b602-49e5c9fbc8f1.mpk successful, with status code 201
2023-07-03 14:40:52,024 fedbiomed INFO - Saved aggregated params for round 6 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_9adf34fb-59ca-4107-b602-49e5c9fbc8f1.mpk
2023-07-03 14:40:52,024 fedbiomed INFO - Sampled nodes in round 7 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:40:52,025 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 7, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_9adf34fb-59ca-4107-b602-49e5c9fbc8f1.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:40:52,025 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:52,026 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 7, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_9adf34fb-59ca-4107-b602-49e5c9fbc8f1.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:40:52,026 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:40:52,069 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 8 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.074448
---------
2023-07-03 14:40:52,070 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 8 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.741083
---------
2023-07-03 14:40:52,110 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 8 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.501979
---------
2023-07-03 14:40:52,111 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 8 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 1.574092
---------
2023-07-03 14:40:52,117 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 8 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 2.381271
---------
2023-07-03 14:40:52,122 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 8 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 2.232035
---------
2023-07-03 14:40:52,141 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 8 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 2.495739
---------
2023-07-03 14:41:02,038 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_809ca19c-99a4-4bda-803a-7d6741b1e7a1.mpk
2023-07-03 14:41:02,042 fedbiomed DEBUG - download of file node_params_8c22eda7-5d1c-4181-8657-afc804b15335.mpk successful, with status code 200
2023-07-03 14:41:02,043 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_3176c29b-8233-432f-beb5-5bc00389e4d2.mpk
2023-07-03 14:41:02,045 fedbiomed DEBUG - download of file node_params_4f79e57f-8f88-4e81-a39d-f40d7d255f8a.mpk successful, with status code 200
2023-07-03 14:41:02,047 fedbiomed INFO - Nodes that successfully reply in round 7 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:41:02,056 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_2b46701a-eb29-461a-bfd1-416223570a76.mpk successful, with status code 201
2023-07-03 14:41:02,056 fedbiomed INFO - Saved aggregated params for round 7 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_2b46701a-eb29-461a-bfd1-416223570a76.mpk
2023-07-03 14:41:02,056 fedbiomed INFO - Sampled nodes in round 8 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:41:02,057 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 8, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_2b46701a-eb29-461a-bfd1-416223570a76.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:41:02,057 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:41:02,058 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 8, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_2b46701a-eb29-461a-bfd1-416223570a76.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:41:02,058 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:41:02,091 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 9 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.330070
---------
2023-07-03 14:41:02,137 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 9 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.796861
---------
2023-07-03 14:41:02,138 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 9 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 1.255121
---------
2023-07-03 14:41:02,139 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 9 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 2.549095
---------
2023-07-03 14:41:02,158 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 9 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 1.943776
---------
2023-07-03 14:41:02,158 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 9 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 0.770924
---------
2023-07-03 14:41:02,198 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 9 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 1.308329
---------
2023-07-03 14:41:12,070 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_b2c70cae-7bf3-407e-979b-5cee3291fc62.mpk
2023-07-03 14:41:12,074 fedbiomed DEBUG - download of file node_params_c248198b-60f0-4f7c-b333-33c5c8de43fa.mpk successful, with status code 200
2023-07-03 14:41:12,075 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_05161a57-768b-42c7-b76f-cc63666e5052.mpk
2023-07-03 14:41:12,077 fedbiomed DEBUG - download of file node_params_5c6b2079-5bf5-4260-9977-f1abbd3b4139.mpk successful, with status code 200
2023-07-03 14:41:12,079 fedbiomed INFO - Nodes that successfully reply in round 8 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:41:12,087 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_9b3da84a-606d-4689-8384-911c4656123d.mpk successful, with status code 201
2023-07-03 14:41:12,088 fedbiomed INFO - Saved aggregated params for round 8 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_9b3da84a-606d-4689-8384-911c4656123d.mpk
2023-07-03 14:41:12,088 fedbiomed INFO - Sampled nodes in round 9 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:41:12,088 fedbiomed INFO - Sending request
To: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 9, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_9b3da84a-606d-4689-8384-911c4656123d.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_6dc88f60-68fb-4bc7-aa01-2f339ea0c08d'}
-----------------------------------------------------------------
2023-07-03 14:41:12,089 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:41:12,090 fedbiomed INFO - Sending request
To: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Request: : Perform training with the arguments: {'researcher_id': 'researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87', 'job_id': 'fbcc50ad-d4a1-4528-b491-4d1a0bbd5b2f', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.0, 'test_on_local_updates': False, 'test_on_global_updates': False, 'test_metric': None, 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 9, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_8b48c219-cb97-4963-b035-4745618fca23.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_9b3da84a-606d-4689-8384-911c4656123d.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_3fffa907-7569-46f0-a2b1-fa7245e42499', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 14:41:12,090 fedbiomed DEBUG - researcher_dba292ff-efe1-40ad-a1c6-1a4d5f6bbd87
2023-07-03 14:41:12,130 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 10 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.886490
---------
2023-07-03 14:41:12,131 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 10 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.704667
---------
2023-07-03 14:41:12,174 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 10 | Iteration: 4/5 (80%) | Samples: 391/640
Loss hinge: 3.202284
---------
2023-07-03 14:41:12,175 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 10 | Iteration: 3/5 (60%) | Samples: 349/640
Loss hinge: 0.827798
---------
2023-07-03 14:41:12,181 fedbiomed INFO - TRAINING
NODE_ID: node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1
Round 10 | Iteration: 5/5 (100%) | Samples: 519/640
Loss hinge: 10.244533
---------
2023-07-03 14:41:12,183 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 10 | Iteration: 4/5 (80%) | Samples: 477/640
Loss hinge: 1.449496
---------
2023-07-03 14:41:12,202 fedbiomed INFO - TRAINING
NODE_ID: node_10797f2f-2524-4595-a1c6-f3c67e03add1
Round 10 | Iteration: 5/5 (100%) | Samples: 605/640
Loss hinge: 3.531648
---------
2023-07-03 14:41:22,102 fedbiomed INFO - Downloading model params after training on node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_5417d5e1-5850-4473-81a3-5c42fdc8c7b7.mpk
2023-07-03 14:41:22,106 fedbiomed DEBUG - download of file node_params_1c0634dc-8f5e-4082-acea-bc297ab46452.mpk successful, with status code 200
2023-07-03 14:41:22,107 fedbiomed INFO - Downloading model params after training on node_10797f2f-2524-4595-a1c6-f3c67e03add1 - from http://localhost:8844/media/uploads/2023/07/03/node_params_cbf71f9b-ecd0-4922-bc78-a7fb7f2eee81.mpk
2023-07-03 14:41:22,110 fedbiomed DEBUG - download of file node_params_bd17ee53-2501-4e5e-a572-115d610dc49d.mpk successful, with status code 200
2023-07-03 14:41:22,111 fedbiomed INFO - Nodes that successfully reply in round 9 ['node_e27ab041-6133-4e2f-b0ce-afa0a0c59fa1', 'node_10797f2f-2524-4595-a1c6-f3c67e03add1']
2023-07-03 14:41:22,123 fedbiomed DEBUG - HTTP POST request of file /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_2ff74778-3640-470d-8657-185b0dd76a3e.mpk successful, with status code 201
2023-07-03 14:41:22,123 fedbiomed INFO - Saved aggregated params for round 9 in /home/jupyter-sharkovsky/fedbiomed/var/experiments/Experiment_0032/aggregated_params_2ff74778-3640-470d-8657-185b0dd76a3e.mpk
10
Task 3: Model Validation #
During federated training, model validation plays a crucial role in assessing performance without a dedicated holdout dataset. Fed-BioMed enables separate model validation on each node after parameter updates, allowing comparison of model performances. Two types of validation can be performed:
one on globally updated parameters before training a round,
another on locally updated parameters after local training is completed on a node.
This helps users evaluate the impact of node-specific training on model improvement.
Here is the list of validation arguments that can be configured.
test_ratio: Ratio of the validation partition of the dataset. The remaining samples will be used for training. By default, it is 0.0.
test_on_global_updates: Boolean value that indicates whether validation will be applied to globally updated (aggregated) parameters (see Figure 1). Default is False
test_on_local_updates: Boolean value that indicates whether validation will be applied to locally updated (trained) parameters (see Figure 1). Default is False
test_metric: One of MetricTypes that indicates which metric will be used for validation. It can be str or an instance of MetricTypes (e.g. MetricTypes.RECALL or RECALL ). If it is None and there isn’t testing_step defined in the training plan (see section: Define Custom Validation Step) default metric will be ACCURACY.
test_metric_args: A dictionary that contains the arguments that will be used for the metric function.
TO_DO:
Initialize a new experiements.
Use the setters to define the validation arguments.
Launch the training and check the validation performances.
exp = Experiment(tags=tags,
model_args=model_args,
training_plan_class=SkLearnClassifierTrainingPlan,
training_args=training_args,
round_limit=rounds,
aggregator=FedAverage(),
node_selection_strategy=None)
exp.set_test_ratio(0.10)
exp.set_test_on_local_updates(True)
exp.set_test_on_global_updates(True)
exp.set_test_metric('F1_SCORE')
2023-07-03 13:09:00,198 fedbiomed INFO - Searching dataset with data tags: ['heart'] for all nodes
2023-07-03 13:09:10,211 fedbiomed INFO - Node selected for training -> node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
2023-07-03 13:09:10,212 fedbiomed INFO - Node selected for training -> node_8641a287-262f-40cb-a01d-0b17f5c76e2e
2023-07-03 13:09:10,214 fedbiomed INFO - Checking data quality of federated datasets...
2023-07-03 13:09:10,215 fedbiomed DEBUG - Using native Sklearn Optimizer
2023-07-03 13:09:10,217 fedbiomed DEBUG - Model file has been saved: /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py
2023-07-03 13:09:10,233 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py successful, with status code 201
2023-07-03 13:09:10,253 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_c63bc355-b053-443c-95cd-25eb2ab66237.mpk successful, with status code 201
2023-07-03 13:09:10,256 fedbiomed DEBUG - Experimentation training_args updated for `job`
2023-07-03 13:09:10,257 fedbiomed DEBUG - Experimentation training_args updated for `job`
2023-07-03 13:09:10,259 fedbiomed DEBUG - Experimentation training_args updated for `job`
2023-07-03 13:09:10,260 fedbiomed DEBUG - Experimentation training_args updated for `job`
('F1_SCORE', {})
exp.training_args()
{'num_updates': 5,
'batch_size': 128,
'dry_run': False,
'optimizer_args': {},
'epochs': None,
'batch_maxnum': None,
'test_ratio': 0.1,
'test_on_local_updates': True,
'test_on_global_updates': True,
'test_metric': 'F1_SCORE',
'test_metric_args': {},
'log_interval': 10,
'fedprox_mu': None,
'use_gpu': False,
'dp_args': None,
'share_persistent_buffers': True}
exp.run()
2023-07-03 13:10:47,594 fedbiomed INFO - Sampled nodes in round 0 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:10:47,595 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 0, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c63bc355-b053-443c-95cd-25eb2ab66237.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:10:47,597 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:10:47,599 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 0, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_c63bc355-b053-443c-95cd-25eb2ab66237.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:10:47,600 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:10:47,628 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:47,660 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:47,701 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:47,703 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.620690
---------
2023-07-03 13:10:47,705 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.000000
---------
2023-07-03 13:10:47,709 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:47,717 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.541667
---------
2023-07-03 13:10:47,763 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.000000
---------
2023-07-03 13:10:47,820 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 0.646654
---------
2023-07-03 13:10:47,848 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 0.874489
---------
2023-07-03 13:10:47,873 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 1.070804
---------
2023-07-03 13:10:47,899 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 0.961074
---------
2023-07-03 13:10:47,940 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 3.185373
---------
2023-07-03 13:10:47,942 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 1 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.654545
---------
2023-07-03 13:10:47,962 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:10:47,965 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 3.339485
---------
2023-07-03 13:10:47,967 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 1 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.000000
---------
2023-07-03 13:10:47,991 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:10:57,619 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_a851d235-e20b-485f-b5f3-ef256fce6e94.mpk
2023-07-03 13:10:57,624 fedbiomed DEBUG - download of file node_params_087d00eb-43dc-4f0a-acf2-06773c51b403.mpk successful, with status code 200
2023-07-03 13:10:57,626 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_bfc19c75-168f-4f5f-b694-a2241d595e72.mpk
2023-07-03 13:10:57,633 fedbiomed DEBUG - download of file node_params_549a8218-017f-4785-92ac-d5bbeda56bf3.mpk successful, with status code 200
2023-07-03 13:10:57,638 fedbiomed INFO - Nodes that successfully reply in round 0 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:10:57,666 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_da8e3b9e-a27a-499e-ad3d-d8a2bd60f2e7.mpk successful, with status code 201
2023-07-03 13:10:57,668 fedbiomed INFO - Saved aggregated params for round 0 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_da8e3b9e-a27a-499e-ad3d-d8a2bd60f2e7.mpk
2023-07-03 13:10:57,669 fedbiomed INFO - Sampled nodes in round 1 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:10:57,671 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 1, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_da8e3b9e-a27a-499e-ad3d-d8a2bd60f2e7.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:10:57,672 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:10:57,674 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 1, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_da8e3b9e-a27a-499e-ad3d-d8a2bd60f2e7.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:10:57,676 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:10:57,711 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:57,751 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:57,756 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:57,758 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.650000
---------
2023-07-03 13:10:57,760 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:10:57,761 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.814815
---------
2023-07-03 13:10:57,799 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.571441
---------
2023-07-03 13:10:57,813 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.826935
---------
2023-07-03 13:10:57,891 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 1.080854
---------
2023-07-03 13:10:57,908 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 5.621247
---------
2023-07-03 13:10:57,948 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 3.503852
---------
2023-07-03 13:10:57,971 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 3.111656
---------
2023-07-03 13:10:58,000 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 3.101825
---------
2023-07-03 13:10:58,006 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 2 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.666667
---------
2023-07-03 13:10:58,036 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 2.136677
---------
2023-07-03 13:10:58,046 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 2 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.769231
---------
2023-07-03 13:10:58,056 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:10:58,077 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:07,696 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_01298da2-5f69-44bd-a3fb-a79d1c3a3b8b.mpk
2023-07-03 13:11:07,703 fedbiomed DEBUG - download of file node_params_f9d00178-e37a-4711-91c1-452364324219.mpk successful, with status code 200
2023-07-03 13:11:07,705 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_4fbd15b3-06d2-4f99-834b-71b026f910be.mpk
2023-07-03 13:11:07,714 fedbiomed DEBUG - download of file node_params_0764d423-ba75-4eec-8fac-e01490bad39c.mpk successful, with status code 200
2023-07-03 13:11:07,718 fedbiomed INFO - Nodes that successfully reply in round 1 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:07,745 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_f2749202-bb4a-4289-8dca-c370b43795ce.mpk successful, with status code 201
2023-07-03 13:11:07,746 fedbiomed INFO - Saved aggregated params for round 1 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_f2749202-bb4a-4289-8dca-c370b43795ce.mpk
2023-07-03 13:11:07,747 fedbiomed INFO - Sampled nodes in round 2 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:07,747 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 2, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_f2749202-bb4a-4289-8dca-c370b43795ce.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:07,748 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:07,750 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 2, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_f2749202-bb4a-4289-8dca-c370b43795ce.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:07,751 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:07,790 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:07,792 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:07,831 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.723404
---------
2023-07-03 13:11:07,833 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:07,834 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:07,837 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.750000
---------
2023-07-03 13:11:07,854 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.847779
---------
2023-07-03 13:11:07,893 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.976905
---------
2023-07-03 13:11:07,952 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 1.424982
---------
2023-07-03 13:11:07,975 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 1.831711
---------
2023-07-03 13:11:08,010 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 0.862112
---------
2023-07-03 13:11:08,042 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 1.656498
---------
2023-07-03 13:11:08,082 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 0.762910
---------
2023-07-03 13:11:08,087 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 3 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.756757
---------
2023-07-03 13:11:08,116 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:08,120 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.686199
---------
2023-07-03 13:11:08,125 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 3 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.666667
---------
2023-07-03 13:11:08,151 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:17,770 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_e2048bb7-4666-4b16-82e2-6b61a4dc9c9b.mpk
2023-07-03 13:11:17,779 fedbiomed DEBUG - download of file node_params_467cd333-c31b-486d-be19-7a330f059e66.mpk successful, with status code 200
2023-07-03 13:11:17,780 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_50e22a77-5eee-473e-a1ff-afc72d6c5317.mpk
2023-07-03 13:11:17,788 fedbiomed DEBUG - download of file node_params_1a214c29-9dab-4e99-a94a-c601964d06c8.mpk successful, with status code 200
2023-07-03 13:11:17,790 fedbiomed INFO - Nodes that successfully reply in round 2 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:17,816 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_b325e05d-1e8d-4806-bb12-b3c95be8fb56.mpk successful, with status code 201
2023-07-03 13:11:17,817 fedbiomed INFO - Saved aggregated params for round 2 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_b325e05d-1e8d-4806-bb12-b3c95be8fb56.mpk
2023-07-03 13:11:17,818 fedbiomed INFO - Sampled nodes in round 3 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:17,818 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 3, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_b325e05d-1e8d-4806-bb12-b3c95be8fb56.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:17,819 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:17,821 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 3, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_b325e05d-1e8d-4806-bb12-b3c95be8fb56.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:17,822 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:17,851 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:17,853 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:17,892 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.880000
---------
2023-07-03 13:11:17,894 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:17,928 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.110141
---------
2023-07-03 13:11:17,930 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:17,932 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.722222
---------
2023-07-03 13:11:17,958 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.788128
---------
2023-07-03 13:11:18,035 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 1.704669
---------
2023-07-03 13:11:18,045 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 2.378920
---------
2023-07-03 13:11:18,097 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 3.010502
---------
2023-07-03 13:11:18,102 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 1.556234
---------
2023-07-03 13:11:18,156 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 3.040663
---------
2023-07-03 13:11:18,160 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 4 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.900000
---------
2023-07-03 13:11:18,162 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.326774
---------
2023-07-03 13:11:18,166 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 4 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.755556
---------
2023-07-03 13:11:18,189 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:18,205 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:27,839 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_3f0dfc90-78b6-44b6-80ff-02e4ec5c1893.mpk
2023-07-03 13:11:27,845 fedbiomed DEBUG - download of file node_params_7b7a3d7f-f590-463c-8254-e41e0d714867.mpk successful, with status code 200
2023-07-03 13:11:27,847 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_fd0a41e2-e711-41a6-816e-aa1cba5257b6.mpk
2023-07-03 13:11:27,855 fedbiomed DEBUG - download of file node_params_c1b91ec8-b0a6-4df4-9298-855defddc08a.mpk successful, with status code 200
2023-07-03 13:11:27,858 fedbiomed INFO - Nodes that successfully reply in round 3 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:27,888 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_184fa830-c29d-411c-a289-0ade92732955.mpk successful, with status code 201
2023-07-03 13:11:27,890 fedbiomed INFO - Saved aggregated params for round 3 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_184fa830-c29d-411c-a289-0ade92732955.mpk
2023-07-03 13:11:27,892 fedbiomed INFO - Sampled nodes in round 4 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:27,893 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 4, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_184fa830-c29d-411c-a289-0ade92732955.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:27,894 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:27,896 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 4, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_184fa830-c29d-411c-a289-0ade92732955.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:27,897 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:27,932 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:27,934 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:27,975 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.840000
---------
2023-07-03 13:11:27,977 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:27,979 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:27,981 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.717949
---------
2023-07-03 13:11:28,017 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.220582
---------
2023-07-03 13:11:28,051 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.791132
---------
2023-07-03 13:11:28,139 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 0.935866
---------
2023-07-03 13:11:28,148 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 1.739728
---------
2023-07-03 13:11:28,203 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 1.499985
---------
2023-07-03 13:11:28,210 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 1.843874
---------
2023-07-03 13:11:28,256 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 1.393408
---------
2023-07-03 13:11:28,257 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 5 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.647059
---------
2023-07-03 13:11:28,259 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.451019
---------
2023-07-03 13:11:28,265 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 5 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.750000
---------
2023-07-03 13:11:28,282 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:28,292 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:37,917 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_5abd9e05-6407-4216-8ac4-e0c14b226afa.mpk
2023-07-03 13:11:37,925 fedbiomed DEBUG - download of file node_params_911e8f78-bc17-46bd-9d4e-f5a072d9db2c.mpk successful, with status code 200
2023-07-03 13:11:37,927 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_107c9cca-8051-4a9d-8c83-095453773d1b.mpk
2023-07-03 13:11:37,936 fedbiomed DEBUG - download of file node_params_d935363b-2f26-4775-ad88-3a76c0997939.mpk successful, with status code 200
2023-07-03 13:11:37,941 fedbiomed INFO - Nodes that successfully reply in round 4 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:37,972 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_149700c9-f09e-4f2b-b59f-3cf8ca9dda9f.mpk successful, with status code 201
2023-07-03 13:11:37,973 fedbiomed INFO - Saved aggregated params for round 4 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_149700c9-f09e-4f2b-b59f-3cf8ca9dda9f.mpk
2023-07-03 13:11:37,974 fedbiomed INFO - Sampled nodes in round 5 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:37,975 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 5, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_149700c9-f09e-4f2b-b59f-3cf8ca9dda9f.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:37,976 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:37,977 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 5, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_149700c9-f09e-4f2b-b59f-3cf8ca9dda9f.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:37,978 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:38,016 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:38,019 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:38,059 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.740741
---------
2023-07-03 13:11:38,060 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:38,062 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:38,064 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.774194
---------
2023-07-03 13:11:38,083 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.168365
---------
2023-07-03 13:11:38,122 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.097516
---------
2023-07-03 13:11:38,187 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 1.427672
---------
2023-07-03 13:11:38,214 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 1.031782
---------
2023-07-03 13:11:38,251 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 2.022977
---------
2023-07-03 13:11:38,270 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 2.552519
---------
2023-07-03 13:11:38,309 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 2.463381
---------
2023-07-03 13:11:38,314 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 6 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.756757
---------
2023-07-03 13:11:38,325 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 3.149643
---------
2023-07-03 13:11:38,327 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 6 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.545455
---------
2023-07-03 13:11:38,337 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:38,347 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:47,996 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_8051ef50-58f1-431b-ba61-fa2a7de4425e.mpk
2023-07-03 13:11:48,003 fedbiomed DEBUG - download of file node_params_eb0d8cf1-796c-441f-be19-3efb7ec9a348.mpk successful, with status code 200
2023-07-03 13:11:48,005 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_23b6ff12-0114-4d39-b611-5407b97a298c.mpk
2023-07-03 13:11:48,014 fedbiomed DEBUG - download of file node_params_96e1f4da-2feb-43d2-b739-aec6f9470a64.mpk successful, with status code 200
2023-07-03 13:11:48,018 fedbiomed INFO - Nodes that successfully reply in round 5 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:48,044 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_1e4efd08-3675-4c18-a8d8-71ee656cf2e4.mpk successful, with status code 201
2023-07-03 13:11:48,045 fedbiomed INFO - Saved aggregated params for round 5 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_1e4efd08-3675-4c18-a8d8-71ee656cf2e4.mpk
2023-07-03 13:11:48,046 fedbiomed INFO - Sampled nodes in round 6 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:48,047 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 6, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_1e4efd08-3675-4c18-a8d8-71ee656cf2e4.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:48,047 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:48,048 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 6, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_1e4efd08-3675-4c18-a8d8-71ee656cf2e4.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:48,049 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:48,075 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:48,077 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:48,112 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.685714
---------
2023-07-03 13:11:48,114 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:48,115 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:48,118 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.722222
---------
2023-07-03 13:11:48,157 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.142721
---------
2023-07-03 13:11:48,193 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.086165
---------
2023-07-03 13:11:48,289 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 0.863901
---------
2023-07-03 13:11:48,304 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 0.980729
---------
2023-07-03 13:11:48,361 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 1.270667
---------
2023-07-03 13:11:48,375 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 1.046473
---------
2023-07-03 13:11:48,436 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 0.967612
---------
2023-07-03 13:11:48,438 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 7 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.652174
---------
2023-07-03 13:11:48,448 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.025577
---------
2023-07-03 13:11:48,449 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 7 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.818182
---------
2023-07-03 13:11:48,463 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:48,473 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:58,062 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_91cb60c3-262f-4462-b47c-46b168b222ad.mpk
2023-07-03 13:11:58,069 fedbiomed DEBUG - download of file node_params_6578107f-179c-4a8e-805f-0b6577afcf02.mpk successful, with status code 200
2023-07-03 13:11:58,071 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_a0330ed0-5d73-4282-890d-f718327c26cf.mpk
2023-07-03 13:11:58,079 fedbiomed DEBUG - download of file node_params_e3b1a95e-5bf4-4db4-9682-32a05dc624d6.mpk successful, with status code 200
2023-07-03 13:11:58,081 fedbiomed INFO - Nodes that successfully reply in round 6 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:58,112 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_ff50f66b-d95f-4c80-b9e2-b9d02395e61c.mpk successful, with status code 201
2023-07-03 13:11:58,113 fedbiomed INFO - Saved aggregated params for round 6 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_ff50f66b-d95f-4c80-b9e2-b9d02395e61c.mpk
2023-07-03 13:11:58,113 fedbiomed INFO - Sampled nodes in round 7 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:11:58,114 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 7, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_ff50f66b-d95f-4c80-b9e2-b9d02395e61c.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:11:58,115 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:58,116 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 7, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_ff50f66b-d95f-4c80-b9e2-b9d02395e61c.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:11:58,117 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:11:58,151 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:58,192 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:58,198 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:58,199 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:11:58,201 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.867925
---------
2023-07-03 13:11:58,205 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.869565
---------
2023-07-03 13:11:58,239 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.966867
---------
2023-07-03 13:11:58,264 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.986017
---------
2023-07-03 13:11:58,341 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 2.760947
---------
2023-07-03 13:11:58,349 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 3.215534
---------
2023-07-03 13:11:58,404 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 2.841887
---------
2023-07-03 13:11:58,419 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 2.925228
---------
2023-07-03 13:11:58,459 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.613500
---------
2023-07-03 13:11:58,461 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 8 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.666667
---------
2023-07-03 13:11:58,469 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 1.259462
---------
2023-07-03 13:11:58,470 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 8 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.837209
---------
2023-07-03 13:11:58,486 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:11:58,497 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:12:08,137 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_dea4b03c-c2bd-440b-9c50-bc8e0ebedd99.mpk
2023-07-03 13:12:08,145 fedbiomed DEBUG - download of file node_params_8a4dcb11-f6c5-4492-ac8f-e2246f3cb971.mpk successful, with status code 200
2023-07-03 13:12:08,147 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_fc32ffcd-2ab3-452b-a6fd-bb8abb558737.mpk
2023-07-03 13:12:08,153 fedbiomed DEBUG - download of file node_params_655c3559-2d88-447a-94d7-f56020623245.mpk successful, with status code 200
2023-07-03 13:12:08,155 fedbiomed INFO - Nodes that successfully reply in round 7 ['node_8641a287-262f-40cb-a01d-0b17f5c76e2e', 'node_1c6bdcb7-99e7-4278-95c9-98caeda6971b']
2023-07-03 13:12:08,178 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_2ed84242-6722-45ef-b979-3e0453616abe.mpk successful, with status code 201
2023-07-03 13:12:08,179 fedbiomed INFO - Saved aggregated params for round 7 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_2ed84242-6722-45ef-b979-3e0453616abe.mpk
2023-07-03 13:12:08,179 fedbiomed INFO - Sampled nodes in round 8 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:12:08,180 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 8, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_2ed84242-6722-45ef-b979-3e0453616abe.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:12:08,180 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:08,181 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 8, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_2ed84242-6722-45ef-b979-3e0453616abe.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:12:08,182 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:08,221 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:08,224 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:08,264 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.727273
---------
2023-07-03 13:12:08,266 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:08,268 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:08,270 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.850000
---------
2023-07-03 13:12:08,307 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.944156
---------
2023-07-03 13:12:08,347 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.995382
---------
2023-07-03 13:12:08,430 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 1.384562
---------
2023-07-03 13:12:08,446 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 1.229974
---------
2023-07-03 13:12:08,487 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 1.127323
---------
2023-07-03 13:12:08,498 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 1.205052
---------
2023-07-03 13:12:08,543 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 0.900732
---------
2023-07-03 13:12:08,546 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 9 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.666667
---------
2023-07-03 13:12:08,563 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.179523
---------
2023-07-03 13:12:08,568 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 9 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.777778
---------
2023-07-03 13:12:08,582 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:12:08,588 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:12:18,201 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_b6c491dc-c946-4799-aaec-584e33a040fa.mpk
2023-07-03 13:12:18,206 fedbiomed DEBUG - download of file node_params_fc646d1c-eaf2-41dc-b155-3413e8ada296.mpk successful, with status code 200
2023-07-03 13:12:18,207 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_b4447156-3406-4cba-8842-bfb2fd27208b.mpk
2023-07-03 13:12:18,213 fedbiomed DEBUG - download of file node_params_23e098aa-aa5a-41c0-8dd6-d72f9a2cdeb2.mpk successful, with status code 200
2023-07-03 13:12:18,215 fedbiomed INFO - Nodes that successfully reply in round 8 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:12:18,239 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_5f0677f1-a85a-4bc3-b213-c607a2030e8d.mpk successful, with status code 201
2023-07-03 13:12:18,240 fedbiomed INFO - Saved aggregated params for round 8 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_5f0677f1-a85a-4bc3-b213-c607a2030e8d.mpk
2023-07-03 13:12:18,241 fedbiomed INFO - Sampled nodes in round 9 ['node_1c6bdcb7-99e7-4278-95c9-98caeda6971b', 'node_8641a287-262f-40cb-a01d-0b17f5c76e2e']
2023-07-03 13:12:18,242 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 9, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_5f0677f1-a85a-4bc3-b213-c607a2030e8d.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_73474707-cfad-4e6a-b585-9a3520fdd776'}
-----------------------------------------------------------------
2023-07-03 13:12:18,243 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:18,246 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: : Perform training with the arguments: {'researcher_id': 'researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e', 'job_id': '598d23e9-d91e-4526-b1b4-aa8ab8b2e30a', 'training_args': {'num_updates': 5, 'batch_size': 128, 'dry_run': False, 'optimizer_args': {}, 'epochs': None, 'batch_maxnum': None, 'test_ratio': 0.1, 'test_on_local_updates': True, 'test_on_global_updates': True, 'test_metric': 'F1_SCORE', 'test_metric_args': {}, 'log_interval': 10, 'fedprox_mu': None, 'use_gpu': False, 'dp_args': None, 'share_persistent_buffers': True}, 'training': True, 'model_args': {'max_iter': 1000, 'tol': 0.1, 'n_features': 18, 'n_classes': 2, 'loss': 'hinge', 'verbose': 1}, 'round': 9, 'secagg_servkey_id': None, 'secagg_biprime_id': None, 'secagg_random': None, 'secagg_clipping_range': None, 'command': 'train', 'training_plan_url': 'http://localhost:8844/media/uploads/2023/07/03/my_model_b511f38a-c219-4bcd-9a43-27bcd5a25797.py', 'params_url': 'http://localhost:8844/media/uploads/2023/07/03/aggregated_params_5f0677f1-a85a-4bc3-b213-c607a2030e8d.mpk', 'training_plan_class': 'SkLearnClassifierTrainingPlan', 'dataset_id': 'dataset_d45526f2-d699-4b64-8934-90d06d77967a', 'protocol_version': '1'}
-----------------------------------------------------------------
2023-07-03 13:12:18,247 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:18,281 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:18,282 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:18,323 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.863636
---------
2023-07-03 13:12:18,326 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:18,357 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:18,359 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.620690
---------
2023-07-03 13:12:18,363 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 1.196435
---------
2023-07-03 13:12:18,395 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 1/5 (20%) | Samples: 128/640
Loss hinge: 0.714835
---------
2023-07-03 13:12:18,466 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 3/5 (60%) | Samples: 314/640
Loss hinge: 1.649812
---------
2023-07-03 13:12:18,467 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 3/5 (60%) | Samples: 351/640
Loss hinge: 4.314792
---------
2023-07-03 13:12:18,512 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 4/5 (80%) | Samples: 442/640
Loss hinge: 3.994241
---------
2023-07-03 13:12:18,516 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 4/5 (80%) | Samples: 479/640
Loss hinge: 1.290517
---------
2023-07-03 13:12:18,560 fedbiomed INFO - TRAINING
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 5/5 (100%) | Samples: 570/640
Loss hinge: 1.708250
---------
2023-07-03 13:12:18,563 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 10 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.750000
---------
2023-07-03 13:12:18,572 fedbiomed INFO - TRAINING
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 5/5 (100%) | Samples: 607/640
Loss hinge: 1.282847
---------
2023-07-03 13:12:18,575 fedbiomed INFO - VALIDATION ON LOCAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 10 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.851064
---------
2023-07-03 13:12:18,595 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:12:18,604 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: results uploaded successfully
-----------------------------------------------------------------
2023-07-03 13:12:28,259 fedbiomed INFO - Downloading model params after training on node_8641a287-262f-40cb-a01d-0b17f5c76e2e - from http://localhost:8844/media/uploads/2023/07/03/node_params_deca3fdb-b149-4302-bd2b-d16ddbbadab8.mpk
2023-07-03 13:12:28,263 fedbiomed DEBUG - download of file node_params_1dd53dca-818d-400e-9975-6ffe7eb6170f.mpk successful, with status code 200
2023-07-03 13:12:28,264 fedbiomed INFO - Downloading model params after training on node_1c6bdcb7-99e7-4278-95c9-98caeda6971b - from http://localhost:8844/media/uploads/2023/07/03/node_params_212cd78e-6078-4b5d-a21e-eef8154f0a37.mpk
2023-07-03 13:12:28,269 fedbiomed DEBUG - download of file node_params_2f5c982f-cc6e-4fc5-94e0-77f159bf3aa5.mpk successful, with status code 200
2023-07-03 13:12:28,271 fedbiomed INFO - Nodes that successfully reply in round 9 ['node_8641a287-262f-40cb-a01d-0b17f5c76e2e', 'node_1c6bdcb7-99e7-4278-95c9-98caeda6971b']
2023-07-03 13:12:28,291 fedbiomed DEBUG - HTTP POST request of file /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_112c956e-b7d7-4fb6-9238-d450adb9770f.mpk successful, with status code 201
2023-07-03 13:12:28,292 fedbiomed INFO - Saved aggregated params for round 9 in /user/linnocen/home/fedbiomed/var/experiments/Experiment_0001/aggregated_params_112c956e-b7d7-4fb6-9238-d450adb9770f.mpk
2023-07-03 13:12:28,293 fedbiomed INFO - Sending request
To: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Request: :Perform final validation on aggregated parameters
-----------------------------------------------------------------
2023-07-03 13:12:28,294 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:28,295 fedbiomed INFO - Sending request
To: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Request: :Perform final validation on aggregated parameters
-----------------------------------------------------------------
2023-07-03 13:12:28,296 fedbiomed DEBUG - researcher_5d3b393d-e248-4e3f-a028-b87fe5fbbe6e
2023-07-03 13:12:28,326 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:28,328 fedbiomed INFO - INFO
NODE node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:28,358 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_1c6bdcb7-99e7-4278-95c9-98caeda6971b
Round 11 | Iteration: 1/1 (100%) | Samples: 40/40
F1_SCORE: 0.812500
---------
2023-07-03 13:12:28,361 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:28,362 fedbiomed INFO - INFO
NODE node_8641a287-262f-40cb-a01d-0b17f5c76e2e
MESSAGE: NPDataLoader expanding 1-dimensional target to become 2-dimensional.
-----------------------------------------------------------------
2023-07-03 13:12:28,398 fedbiomed INFO - VALIDATION ON GLOBAL UPDATES
NODE_ID: node_8641a287-262f-40cb-a01d-0b17f5c76e2e
Round 11 | Iteration: 1/1 (100%) | Samples: 35/35
F1_SCORE: 0.750000
---------
10