bug: screenshot is not loaded with Espresso driver

[zhu-xiaowei]

Is this an issue specifically with Appium Inspector?

• I have verified that my issue does not occur with Appium, and should be investigated as an Appium Inspector issue

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

capabilities:

{
  "platformName": "Android",
  "appium:automationName": "espresso",
  "appium:deviceName": "Android",
  "appium:appPackage": "com.xxx.xxx",
  "appium:appActivity": ".core.presentation.MainActivity",
  "appium:language": "en",
  "appium:locale": "US",
  "appium:skipUnlock": true,
  "appium:fullReset": false,
  "appium:forceEspressoRebuild": true,
  "appium:showGradleLog": true,
  "appium:espressoBuildConfig": "{\"toolsVersions\":{\"gradle\":\"7.4\",\"androidGradlePlugin\":\"7.3.1\",\"compileSdk\":\"34\",\"minSdk\":\"21\",\"targetSdk\":\"34\",\"composeVersion\":\"1.4.3\",\"espressoVersion\":\"3.5.1\",\"kotlin\":\"1.8.10\"},\"additionalAndroidTestDependencies\":[\"androidx.lifecycle:lifecycle-common:2.6.2\",\"androidx.activity:activity:1.9.0-alpha01\",\"androidx.fragment:fragment:1.5.1\",\"androidx.test.ext:junit:1.1.3\",\"androidx.test.espresso:espresso-core:3.4.0\",\"androidx.compose.ui:ui-test-junit4:1.4.3\",\"androidx.compose.ui:ui:1.4.3\",\"androidx.activity:activity-compose:1.9.0-alpha01\",\"androidx.compose.material:material:1.4.3\"]}"
}

Expected Behavior

load the device screen image

Operating System

Mac

Appium Inspector Version

2023.12.2

Appium Version

2.3.0

Further Information

No response

[eglitise]

Thanks for the report. Could you please provide the full Appium server log? (Gradle log can probably be excluded)
Also, do you maybe know if this was working on a previous Inspector version?

[eglitise]

Closing due to no response

[bronik]

facing same issue
appium: 2.4.1
inspector gui: 2023.12.2
java: 17

capabilities:
{
"platformName": "android",
"appium:automationName": "Espresso",
"appium:app": "app",
"appium:showGradleLog": true,
"appium:noReset": true,
"appium:allowTestPackages": true,
"appium:espressoBuildConfig": "{"toolsVersions": {"composeVersion":"1.4.1", "kotlin":"1.8.20", "gradle": "7.6", "androidGradlePlugin":"7.3.1", "compileSdk": "33", "targetSdk": "31", "minSdk": "24", "annotationVersion":"1.1.0", "espressoVersion":"3.5.1"}, "additionalAppDependencies": ["org.jetbrains.kotlin:kotlin-stdlib:1.8.20"], "additionalAndroidTestDependencies": ["org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.3"]}",
"appium:platformVersion": "14.0",
"appium:appWaitActivity": "app"
}

appium logs:
https://gist.github.com/bronik/6cde1fdf9836a82286bb075afee80516

can anyone help me with this?

[eglitise]

@bronik thanks for the log. Could you please check this with Appium on its own? That is, create a small test using your Appium client that tries to take a screenshot (the method name will depend on your client language).

[bronik]

@eglitise if I got you right:
https://gist.github.com/bronik/47b091ecf2d4a6589bcf0785f24b6a4b

i'm using: appium java-client 9.0.0

and, of course, if u need some addition info - just let me know)

[eglitise]

@bronik thanks, but this snippet is not enough, could you please paste the full log?

[bronik]

@eglitise
https://gist.github.com/bronik/a5f93b03fa26db90de0e0f536c21f040

test body:

• click on element on UI
• get screenshot as Base64 (+ assert not null)
• get screenshot as File (+ assert not null)
• get screenshot as Bytes (+ assert not null)

[fleytman]

There was no bug on espresso driver version 2.25.9, after upgrading to the latest version 2.32.2 the bug started to appear

[fleytman]

logcat on 2.25.9 espresso driver

01-11 19:41:12.369 14746 14747 I Gralloc4: mapper 4.x is not supported
01-11 19:41:12.374 14746 14747 D HostConnection: createUnique: call
01-11 19:41:12.374 14746 14747 D HostConnection: HostConnection::get() New Host Connection established 0xb40000712c249fd0, tid 14747
01-11 19:41:12.375 14746 14747 D goldfish-address-space: allocate: Ask for block of size 0x100
01-11 19:41:12.375 14746 14747 D goldfish-address-space: allocate: ioctl allocate returned offset 0x1fbff8000 size 0x8000
01-11 19:41:12.382 14746 14747 D HostConnection: HostComposition ext ANDROID_EMU_CHECKSUM_HELPER_v1 ANDROID_EMU_native_sync_v2 ANDROID_EMU_native_sync_v3 ANDROID_EMU_native_sync_v4 ANDROID_EMU_dma_v1 ANDROID_EMU_direct_mem ANDROID_EMU_vulkan ANDROID_EMU_deferred_vulkan_commands ANDROID_EMU_vulkan_null_optional_strings ANDROID_EMU_vulkan_create_resources_with_requirements ANDROID_EMU_YUV_Cache ANDROID_EMU_vulkan_ignored_handles ANDROID_EMU_has_shared_slots_host_memory_allocator ANDROID_EMU_vulkan_free_memory_sync ANDROID_EMU_vulkan_shader_float16_int8 ANDROID_EMU_vulkan_async_queue_submit ANDROID_EMU_vulkan_queue_submit_with_commands ANDROID_EMU_sync_buffer_data ANDROID_EMU_vulkan_async_qsri ANDROID_EMU_read_color_buffer_dma GL_OES_EGL_image_external_essl3 GL_OES_vertex_array_object GL_KHR_texture_compression_astc_ldr ANDROID_EMU_host_side_tracing ANDROID_EMU_gles_max_version_3_0
01-11 19:41:12.417 12379 14738 D appium  : Started processing GET request for '/session/6d325d13-b0a2-4464-98ff-b9eba468ec6e/window/rect'
01-11 19:41:12.418 12379 14738 D appium  : Matched route definition: class io.appium.espressoserver.lib.http.RouteDefinition
01-11 19:41:12.418 12379 14738 D appium  : Matched route handler: class io.appium.espressoserver.lib.handlers.GetWindowRect
01-11 19:41:12.418 12379 14738 I appium  : Executing GetWindowRect handler
01-11 19:41:12.419 12379 14738 D appium  : Finished processing GET request for '/session/6d325d13-b0a2-4464-98ff-b9eba468ec6e/window/rect'
01-11 19:41:12.420 12379 14738 I appium  : Responding to server with value: WindowRect(width=360, height=800, x=0, y=0)
01-11 19:41:12.430 12379 14738 D appium  : Started processing GET request for '/session/6d325d13-b0a2-4464-98ff-b9eba468ec6e/source'
01-11 19:41:12.430 12379 14738 D appium  : Matched route definition: class io.appium.espressoserver.lib.http.RouteDefinition
01-11 19:41:12.431 12379 14738 D appium  : Matched route handler: class io.appium.espressoserver.lib.handlers.Source
01-11 19:41:12.432 12379 14738 I appium  : Executing Source handler
01-11 19:41:12.435 12379 12379 I ViewInteraction: Performing 'getting view in application' action on view view.getRootView() to equal view
01-11 19:41:12.455 12379 14738 I appium  : The source XML tree has been fetched in 23ms using ByteArrayOutputStream
01-11 19:41:12.456 12379 14738 D appium  : Finished processing GET request for '/session/6d325d13-b0a2-4464-98ff-b9eba468ec6e/source'
01-11 19:41:12.456 12379 14738 I appium  : Responding to server with value: <?xml version='1.0' encoding='UTF-8' standalone='yes' ?><com.android.internal.policy.DecorView index="0" package="com.my.app" class="com.android.internal.policy.DecorView" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="fal…

logcat on 2.32.2

01-12 18:05:34.823 23625 24053 D appium  : Matched route definition: class io.appium.espressoserver.lib.http.RouteDefinition
01-12 18:05:34.823 23625 24053 D appium  : Matched route handler: class io.appium.espressoserver.lib.handlers.GetWindowRect
01-12 18:05:34.824 23625 24053 I appium  : Executing GetWindowRect handler
01-12 18:05:34.830 23625 24053 D appium  : Finished processing GET request for '/session/c1038a10-9446-463f-a8a9-829bacfbe6c6/window/rect'
01-12 18:05:34.830 23625 24053 I appium  : Responding to server with value: WindowRect(width=360, height=800, x=0, y=0)
01-12 18:05:34.864 23625 24053 D appium  : Started processing GET request for '/session/c1038a10-9446-463f-a8a9-829bacfbe6c6/source'
01-12 18:05:34.866 23625 24053 D appium  : Matched route definition: class io.appium.espressoserver.lib.http.RouteDefinition
01-12 18:05:34.866 23625 24053 D appium  : Matched route handler: class io.appium.espressoserver.lib.handlers.Source
01-12 18:05:34.866 23625 24053 I appium  : Executing Source handler
01-12 18:05:34.877 23625 23625 I ViewInteraction: Performing 'getting view in application' action on view view.getRootView() to equal view
01-12 18:05:34.908 23625 24053 I appium  : The source XML tree has been fetched in 40ms using ByteArrayOutputStream
01-12 18:05:34.908 23625 24053 D appium  : Finished processing GET request for '/session/c1038a10-9446-463f-a8a9-829bacfbe6c6/source'
01-12 18:05:34.908 23625 24053 I appium  : Responding to server with value: <?xml version='1.0' encoding='UTF-8' standalone='yes' ?><com.android.internal.policy.DecorView index="0" package="com.my.app" class="com.android.internal.policy.DecorView" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="fal…

[eglitise]

Ok, I've identified that the problem started with Espresso driver 2.26.0, after bumping appium-android-driver to v7, which removed its getScreenshot method.
@mykola-mokhnach @KazuCocoa What would be the better solution here? Adding getScreenshot to the Espresso driver, or adding handling for mobile: screenshots in the Inspector (which is not supported by the xcuitest driver)? I would prefer the former, because the latter would still prevent inspection of Android <10 devices.
Although, I suppose mobile: screenshots would still be necessary for #1294.

[KazuCocoa]

Hm, usually /session/:sessionId/screenshot request goes to the Espresso server running on the device. Removed code from the appium-android-driver you mentioned was maybe:

  async getScreenshot() {
    throw new errors.NotImplementedError('Not implemented');
  },

https://github.com/appium/appium-android-driver/pull/901/files#diff-6a11c8185e3899960e8e3e3c345d13cc69dba0cd9073e1c71b2f957402860a5cL233 ?

[KazuCocoa]

Oh, okay.. the capabilities set by appium inspector has "appium:nativeWebScreenshot": true, which proxy the screenshot endpoint call to chromedriver https://github.com/appium/appium-espresso-driver/blob/619d177700084c3916b569397bac922d00d0cf19/lib/driver.ts#L686-L690 but it should not work for native app only case

[eglitise]

Yes, getScreenshot is the exact method called by the Inspector. UiAutomator2 just overrides it, so it did not have this problem.

[KazuCocoa]

I explored a few. Actually espresso driver should proxy the request to the espresso server with nativeWebScreenshot: true as well. appium/appium-espresso-driver#978 fixes it. The appium-android-driver maybe did not remove getScreenshot from this driver, and the original implementation was maybe throw new errors.NotImplementedError('Not implemented'); thus I'm not sure where affected before, but the PR implements the method in the espresso driver.

[KazuCocoa]

Espresso driver 2.34.4 has the fix with the capability