You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
323 lines
9.4 KiB
323 lines
9.4 KiB
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# Data Science in the Cloud: The \"Azure ML SDK\" way \n",
|
|
"\n",
|
|
"## Introduction\n",
|
|
"\n",
|
|
"In this notebook, we will explore how to use the Azure ML SDK to train, deploy, and utilize a model through Azure ML.\n",
|
|
"\n",
|
|
"Prerequisites:\n",
|
|
"1. You have created an Azure ML workspace.\n",
|
|
"2. You have uploaded the [Heart Failure dataset](https://www.kaggle.com/andrewmvd/heart-failure-clinical-data) into Azure ML.\n",
|
|
"3. You have added this notebook to Azure ML Studio.\n",
|
|
"\n",
|
|
"The steps to follow are:\n",
|
|
"\n",
|
|
"1. Create an Experiment in an existing Workspace.\n",
|
|
"2. Set up a Compute cluster.\n",
|
|
"3. Load the dataset.\n",
|
|
"4. Configure AutoML using AutoMLConfig.\n",
|
|
"5. Execute the AutoML experiment.\n",
|
|
"6. Review the results and identify the best model.\n",
|
|
"7. Register the best model.\n",
|
|
"8. Deploy the best model.\n",
|
|
"9. Use the endpoint.\n",
|
|
"\n",
|
|
"## Azure Machine Learning SDK-specific imports\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"from azureml.core import Workspace, Experiment\n",
|
|
"from azureml.core.compute import AmlCompute\n",
|
|
"from azureml.train.automl import AutoMLConfig\n",
|
|
"from azureml.widgets import RunDetails\n",
|
|
"from azureml.core.model import InferenceConfig, Model\n",
|
|
"from azureml.core.webservice import AciWebservice"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Initialize Workspace\n",
|
|
"Initialize a workspace object using the saved configuration. Ensure the config file is located at .\\config.json\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"ws = Workspace.from_config()\n",
|
|
"print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Create an Azure ML experiment\n",
|
|
"\n",
|
|
"Let's create an experiment named 'aml-experiment' in the workspace we just initialized.\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"experiment_name = 'aml-experiment'\n",
|
|
"experiment = Experiment(ws, experiment_name)\n",
|
|
"experiment"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Create a Compute Cluster\n",
|
|
"You need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/concept-azure-machine-learning-architecture#compute-target) for your AutoML run.\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"aml_name = \"heart-f-cluster\"\n",
|
|
"try:\n",
|
|
" aml_compute = AmlCompute(ws, aml_name)\n",
|
|
" print('Found existing AML compute context.')\n",
|
|
"except:\n",
|
|
" print('Creating new AML compute context.')\n",
|
|
" aml_config = AmlCompute.provisioning_configuration(vm_size = \"Standard_D2_v2\", min_nodes=1, max_nodes=3)\n",
|
|
" aml_compute = AmlCompute.create(ws, name = aml_name, provisioning_configuration = aml_config)\n",
|
|
" aml_compute.wait_for_completion(show_output = True)\n",
|
|
"\n",
|
|
"cts = ws.compute_targets\n",
|
|
"compute_target = cts[aml_name]"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Data\n",
|
|
"Ensure that the dataset has been uploaded to Azure ML and that the key matches the dataset name exactly.\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"key = 'heart-failure-records'\n",
|
|
"dataset = ws.datasets[key]\n",
|
|
"df = dataset.to_pandas_dataframe()\n",
|
|
"df.describe()"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## AutoML Configuration\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"automl_settings = {\n",
|
|
" \"experiment_timeout_minutes\": 20,\n",
|
|
" \"max_concurrent_iterations\": 3,\n",
|
|
" \"primary_metric\" : 'AUC_weighted'\n",
|
|
"}\n",
|
|
"\n",
|
|
"automl_config = AutoMLConfig(compute_target=compute_target,\n",
|
|
" task = \"classification\",\n",
|
|
" training_data=dataset,\n",
|
|
" label_column_name=\"DEATH_EVENT\",\n",
|
|
" enable_early_stopping= True,\n",
|
|
" featurization= 'auto',\n",
|
|
" debug_log = \"automl_errors.log\",\n",
|
|
" **automl_settings\n",
|
|
" )"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## AutoML Run\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"remote_run = experiment.submit(automl_config)"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"RunDetails(remote_run).show()"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"best_run, fitted_model = remote_run.get_output()"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"best_run.get_properties()"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"model_name = best_run.properties['model_name']\n",
|
|
"script_file_name = 'inference/score.py'\n",
|
|
"best_run.download_file('outputs/scoring_file_v_1_0_0.py', 'inference/score.py')\n",
|
|
"description = \"aml heart failure project sdk\"\n",
|
|
"model = best_run.register_model(model_name = model_name,\n",
|
|
" description = description,\n",
|
|
" tags = None)"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Deploy the Best Model\n",
|
|
"\n",
|
|
"Run the following code to deploy the best model. You can check the deployment status in the Azure ML portal. This process may take a few minutes.\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"inference_config = InferenceConfig(entry_script=script_file_name, environment=best_run.get_environment())\n",
|
|
"\n",
|
|
"aciconfig = AciWebservice.deploy_configuration(cpu_cores = 1,\n",
|
|
" memory_gb = 1,\n",
|
|
" tags = {'type': \"automl-heart-failure-prediction\"},\n",
|
|
" description = 'Sample service for AutoML Heart Failure Prediction')\n",
|
|
"\n",
|
|
"aci_service_name = 'automl-hf-sdk'\n",
|
|
"aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aciconfig)\n",
|
|
"aci_service.wait_for_deployment(True)\n",
|
|
"print(aci_service.state)"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Use the Endpoint\n",
|
|
"You can provide inputs based on the sample input below.\n"
|
|
],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"data = {\n",
|
|
" \"data\":\n",
|
|
" [\n",
|
|
" {\n",
|
|
" 'age': \"60\",\n",
|
|
" 'anaemia': \"false\",\n",
|
|
" 'creatinine_phosphokinase': \"500\",\n",
|
|
" 'diabetes': \"false\",\n",
|
|
" 'ejection_fraction': \"38\",\n",
|
|
" 'high_blood_pressure': \"false\",\n",
|
|
" 'platelets': \"260000\",\n",
|
|
" 'serum_creatinine': \"1.40\",\n",
|
|
" 'serum_sodium': \"137\",\n",
|
|
" 'sex': \"false\",\n",
|
|
" 'smoking': \"false\",\n",
|
|
" 'time': \"130\",\n",
|
|
" },\n",
|
|
" ],\n",
|
|
"}\n",
|
|
"\n",
|
|
"test_sample = str.encode(json.dumps(data))"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"source": [
|
|
"response = aci_service.run(input_data=test_sample)\n",
|
|
"response"
|
|
],
|
|
"outputs": [],
|
|
"metadata": {}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n---\n\n**Disclaimer**: \nThis document has been translated using the AI translation service [Co-op Translator](https://github.com/Azure/co-op-translator). While we aim for accuracy, please note that automated translations may include errors or inaccuracies. The original document in its native language should be regarded as the authoritative source. For critical information, professional human translation is advised. We are not responsible for any misunderstandings or misinterpretations resulting from the use of this translation.\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"orig_nbformat": 4,
|
|
"language_info": {
|
|
"name": "python"
|
|
},
|
|
"coopTranslator": {
|
|
"original_hash": "af42669556d5dc19fc4cc3866f7d2597",
|
|
"translation_date": "2025-09-03T20:36:29+00:00",
|
|
"source_file": "5-Data-Science-In-Cloud/19-Azure/notebook.ipynb",
|
|
"language_code": "en"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
} |