Nightly builds with baseline profiles (#1173)
* Overall build * Enable nightly builds * Enable Startup Profile generation for release builds * Skip BP generation for regular Builds * Test baseline profiles for all variants * Cleanup * Remove stale baseline-prof.txt * Add generated profiles to .gitignore * GMD * Setup GMD in separate step * Add GMD setup to all workflows * Lower GMD specs * Update GMD and Android SDK setup * Add options test options for GMD startup * Ensure only one device is used for bp / benchmark * Apply guidance from issuetracker b/287312019 * Add new metrics for baseline profile measurement Added custom metrics to better understand how effective a baseline profile is. These TraceSectionMetrics keep track of JIT compilations as well as class initializations which should go down when a BP is properly applied to the app.pull/1617/head
Ben Weiss
3 months ago
committed by
GitHub
parent
f13324f13a
commit
c01a129e4a
@ -0,0 +1,43 @@
|
||||
name: NightlyBaselineProfiles
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '42 4 * * *'
|
||||
|
||||
jobs:
|
||||
baseline_profiles:
|
||||
name: "Generate Baseline Profiles"
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
timeout-minutes: 60
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Validate Gradle Wrapper
|
||||
uses: gradle/wrapper-validation-action@v1
|
||||
|
||||
- name: Copy CI gradle.properties
|
||||
run: mkdir -p ~/.gradle ; cp .github/ci-gradle.properties ~/.gradle/gradle.properties
|
||||
|
||||
- name: Set up JDK 17
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: 17
|
||||
|
||||
- name: Setup GMD
|
||||
run: ./gradlew :benchmarks:pixel6Api33Setup
|
||||
--info
|
||||
-Pandroid.experimental.testOptions.managedDevices.emulator.showKernelLogging=true
|
||||
-Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
|
||||
|
||||
- name: Build all build type and flavor permutations including baseline profiles
|
||||
run: ./gradlew :app:assemble
|
||||
-Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=baselineprofile
|
||||
-Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
|
||||
-Pandroid.experimental.testOptions.managedDevices.emulator.showKernelLogging=true
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,50 @@
|
||||
/*
|
||||
* Copyright 2024 The Android Open Source Project
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.google.samples.apps.nowinandroid
|
||||
|
||||
import androidx.benchmark.macro.ExperimentalMetricApi
|
||||
import androidx.benchmark.macro.StartupTimingMetric
|
||||
import androidx.benchmark.macro.TraceSectionMetric
|
||||
|
||||
/**
|
||||
* Custom Metrics to measure baseline profile effectiveness.
|
||||
*/
|
||||
class BaselineProfileMetrics {
|
||||
companion object {
|
||||
/**
|
||||
* A [TraceSectionMetric] that tracks the time spent in JIT compilation.
|
||||
*
|
||||
* This number should go down when a baseline profile is applied properly.
|
||||
*/
|
||||
@OptIn(ExperimentalMetricApi::class)
|
||||
val jitCompilationMetric = TraceSectionMetric("JIT Compiling %", label = "JIT compilation")
|
||||
|
||||
/**
|
||||
* A [TraceSectionMetric] that tracks the time spent in class initialization.
|
||||
*
|
||||
* This number should go down when a baseline profile is applied properly.
|
||||
*/
|
||||
@OptIn(ExperimentalMetricApi::class)
|
||||
val classInitMetric = TraceSectionMetric("L%/%;", label = "ClassInit")
|
||||
|
||||
/**
|
||||
* Metrics relevant to startup and baseline profile effectiveness measurement.
|
||||
*/
|
||||
@OptIn(ExperimentalMetricApi::class)
|
||||
val allMetrics = listOf(StartupTimingMetric(), jitCompilationMetric, classInitMetric)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in new issue