Android(安卓)P 图形显示系统(十) BufferQueue(一)
文章目录
- BufferQueue
- Native应用绘制界面
- SurfaceFlinger创建Layer
- Native窗口
- ANativeWindow
- Surface
- SurfaceControl
- 窗口销毁的善后处理
BufferQueue
前面结合应用中WindowSurfaceWrapper的,讲解了应用怎么和SurfaceFlinger建立连接,进行交互的。
BufferQueue 类是 Android 中所有图形处理操作的核心。它的作用很简单:将生成图形数据缓冲区的一方(生产方)连接到接受数据以进行显示或进一步处理的一方(消耗方)。几乎所有在系统中移动图形数据缓冲区的内容都依赖于 BufferQueue。
现在,我们添加一个空的drawNativeWindow实现,先将我们的应用跑起来吧。
int drawNativeWindow(sp /* windowSurface */) { return NO_ERROR;}int main(int argc, char *argv[]) { unsigned samples = 0; printf("usage: %s [samples]\n", argv[0]); if (argc == 2) { samples = atoi( argv[1] ); printf("Multisample enabled: GL_SAMPLES = %u\n", samples); } sp proc(ProcessState::self()); ProcessState::self()->startThreadPool(); sp windowSurface(new WindowSurfaceWrapper(String8("NativeBinApp"))); drawNativeWindow(windowSurface); IPCThreadState::self()->joinThreadPool(); return EXIT_SUCCESS;}
Android.bp如下:
cc_binary { name: "NativeApp", srcs: [ "NativeApp.cpp", "WindowSurfaceWrapper.cpp", ], shared_libs: [ "liblog", "libbinder", "libui", "libgui", "libutils", ], init_rc: ["NativeApp.rc"],}
NativeApp.rc文件如下:
service NativeApp /system/bin/NativeApp class core oneshot
将应用push到系统的bin目录下就可以运行了:
adb push out/target/product/generic/system/bin/NativeApp /vendor/bin/
运行应用:
adb shell NativeBin
很遗憾的是,我们在手机屏幕上是看不是任何东西的。Why?因为没有画任何东西。但是,我们dump SurfaceFlinger的时候,还是能够看到我们创建的应用窗口的,只是没有内容,SurfaceFlinger不显示,即使去显示,也看不到。
adb shell dumpsys SurfaceFlinger
我们创建应用Layer时,名字为“NativeBinApp”,下面就是我们dump出来的Layer的信息:
+ Layer 0x7b3ba66000 (NativeBinApp#0) Region transparentRegion (this=0x7b3ba66380, count=1) [ 0, 0, 0, 0] Region visibleRegion (this=0x7b3ba66010, count=1) [ 0, 0, 0, 0] Region surfaceDamageRegion (this=0x7b3ba66088, count=1) [ 0, 0, 0, 0] layerStack= 0, z=2147483647, pos=(0,0), size=( 720,1280), crop=( 0, 0, -1, -1), finalCrop=( 0, 0, -1, -1), isOpaque=0, invalidate=0, dataspace=Default (0), pixelformat=Unknown/None alpha=1.000, flags=0x00000002, tr=[1.00, 0.00][0.00, 1.00] client=0x7b4002d0c0 format= 2, activeBuffer=[ 0x 0: 0, 0], queued-frames=0, mRefreshPending=0 mTexName=7 mCurrentTexture=-1 mCurrentCrop=[0,0,0,0] mCurrentTransform=0 mAbandoned=0 - BufferQueue mMaxAcquiredBufferCount=1 mMaxDequeuedBufferCount=2 mDequeueBufferCannotBlock=0 mAsyncMode=0 default-size=[720x1280] default-format=2 transform-hint=00 frame-counter=0 FIFO(0): Slots: [00:0x0] state=FREE [01:0x0] state=FREE [02:0x0] state=FREE
我们已经创建了窗口,但是界面没有内容显示,我们先完善我们的应用,在窗口中显示点内容吧~
Native应用绘制界面
下面是drawNativeWindow窗口的对应的代码,关键的步骤的用序号标出来了~
int drawNativeWindow(sp windowSurface) { status_t err = NO_ERROR; ANativeWindowBuffer *aNativeBuffer = nullptr; sp surfaceControl = windowSurface->getSurfaceControl(); ANativeWindow* aNativeWindow = surfaceControl->getSurface().get(); // 1. We need to reconnect to the ANativeWindow as a CPU client to ensure that no frames // get dropped by SurfaceFlinger assuming that these are other frames. err = native_window_api_disconnect(aNativeWindow, NATIVE_WINDOW_API_CPU); if (err != NO_ERROR) { ALOGE("ERROR: unable to native_window_api_disconnect ignore...\n"); } // 2. connect the ANativeWindow as a CPU client err = native_window_api_connect(aNativeWindow, NATIVE_WINDOW_API_CPU); if (err != NO_ERROR) { ALOGE("ERROR: unable to native_window_api_connect\n"); return EXIT_FAILURE; } // 3. set the ANativeWindow dimensions err = native_window_set_buffers_user_dimensions(aNativeWindow, windowSurface->width(), windowSurface->height()); if (err != NO_ERROR) { ALOGE("ERROR: unable to native_window_set_buffers_user_dimensions\n"); return EXIT_FAILURE; } // 4. set the ANativeWindow format int format = PIXEL_FORMAT_RGBX_8888; err = native_window_set_buffers_format(aNativeWindow,format ); if (err != NO_ERROR) { ALOGE("ERROR: unable to native_window_set_buffers_format\n"); return EXIT_FAILURE; } // 5. set the ANativeWindow transform int rotation = 0; int transform = 0; if ((rotation % 90) == 0) { switch ((rotation / 90) & 3) { case 1: transform = HAL_TRANSFORM_ROT_90; break; case 2: transform = HAL_TRANSFORM_ROT_180; break; case 3: transform = HAL_TRANSFORM_ROT_270; break; default: transform = 0; break; } } err = native_window_set_buffers_transform(aNativeWindow, transform); if (err != NO_ERROR) { ALOGE("native_window_set_buffers_transform failed: %s (%d)", strerror(-err), -err); return err; } // 6. handle the ANativeWindow usage int consumerUsage = 0; err = aNativeWindow->query(aNativeWindow, NATIVE_WINDOW_CONSUMER_USAGE_BITS, &consumerUsage); if (err != NO_ERROR) { ALOGE("failed to get consumer usage bits. ignoring"); err = NO_ERROR; } // Make sure to check whether either requested protected buffers. int usage = GRALLOC_USAGE_SW_WRITE_OFTEN; if (usage & GRALLOC_USAGE_PROTECTED) { // Check if the ANativeWindow sends images directly to SurfaceFlinger. int queuesToNativeWindow = 0; err = aNativeWindow->query( aNativeWindow, NATIVE_WINDOW_QUEUES_TO_WINDOW_COMPOSER, &queuesToNativeWindow); if (err != NO_ERROR) { ALOGE("error authenticating native window: %s (%d)", strerror(-err), -err); return err; } // Check if the consumer end of the ANativeWindow can handle protected content. int isConsumerProtected = 0; err = aNativeWindow->query( aNativeWindow, NATIVE_WINDOW_CONSUMER_IS_PROTECTED, &isConsumerProtected); if (err != NO_ERROR) { ALOGE("error query native window: %s (%d)", strerror(-err), -err); return err; } // Deny queuing into native window if neither condition is satisfied. if (queuesToNativeWindow != 1 && isConsumerProtected != 1) { ALOGE("native window cannot handle protected buffers: the consumer should either be " "a hardware composer or support hardware protection"); return PERMISSION_DENIED; } } // 7. set the ANativeWindow usage int finalUsage = usage | consumerUsage; ALOGE("gralloc usage: %#x(producer) + %#x(consumer) = %#x", usage, consumerUsage, finalUsage); err = native_window_set_usage(aNativeWindow, finalUsage); if (err != NO_ERROR) { ALOGE("native_window_set_usage failed: %s (%d)", strerror(-err), -err); return err; } // 8. set the ANativeWindow scale mode err = native_window_set_scaling_mode( aNativeWindow, NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW); if (err != NO_ERROR) { ALOGE("native_window_set_scaling_mode failed: %s (%d)", strerror(-err), -err); return err; } ALOGE("set up nativeWindow %p for %dx%d, color %#x, rotation %d, usage %#x", aNativeWindow, windowSurface->width(), windowSurface->height(), format, rotation, finalUsage); // 9. set the ANativeWindow permission to allocte new buffer, default is true static_cast(aNativeWindow)->getIGraphicBufferProducer()->allowAllocation(true); // 10. set the ANativeWindow buffer count int numBufs = 0; int minUndequeuedBufs = 0; err = aNativeWindow->query(aNativeWindow, NATIVE_WINDOW_MIN_UNDEQUEUED_BUFFERS, &minUndequeuedBufs); if (err != NO_ERROR) { ALOGE("error pushing blank frames: MIN_UNDEQUEUED_BUFFERS query " "failed: %s (%d)", strerror(-err), -err); goto handle_error; } numBufs = minUndequeuedBufs + 1; err = native_window_set_buffer_count(aNativeWindow, numBufs); if (err != NO_ERROR) { ALOGE("error pushing blank frames: set_buffer_count failed: %s (%d)", strerror(-err), -err); goto handle_error; } // 11. draw the ANativeWindow for (int i = 0; i < numBufs + 1; i++) { // 12. dequeue a buffer int hwcFD= -1; err = aNativeWindow->dequeueBuffer(aNativeWindow, &aNativeBuffer, &hwcFD); if (err != NO_ERROR) { ALOGE("error pushing blank frames: dequeueBuffer failed: %s (%d)", strerror(-err), -err); break; } // 13. make sure really control the dequeued buffer sp hwcFence(new Fence(hwcFD)); int waitResult = hwcFence->waitForever("dequeueBuffer_EmptyNative"); if (waitResult != OK) { ALOGE("dequeueBuffer_EmptyNative: Fence::wait returned an error: %d", waitResult); break; } sp buf(GraphicBuffer::from(aNativeBuffer)); // 14. Fill the buffer with black uint8_t *img = NULL; err = buf->lock(GRALLOC_USAGE_SW_WRITE_OFTEN, (void**)(&img)); if (err != NO_ERROR) { ALOGE("error pushing blank frames: lock failed: %s (%d)", strerror(-err), -err); break; } //15. Draw the window, here we fill the window with black. *img = 0; err = buf->unlock(); if (err != NO_ERROR) { ALOGE("error pushing blank frames: unlock failed: %s (%d)", strerror(-err), -err); break; } // 16. queue the buffer to display int gpuFD = -1; err = aNativeWindow->queueBuffer(aNativeWindow, buf->getNativeBuffer(), gpuFD); if (err != NO_ERROR) { ALOGE("error pushing blank frames: queueBuffer failed: %s (%d)", strerror(-err), -err); break; } aNativeBuffer = NULL; }handle_error: // 17. cancel buffer if (aNativeBuffer != NULL) { aNativeWindow->cancelBuffer(aNativeWindow, aNativeBuffer, -1); aNativeBuffer = NULL; } // 18. Clean up after success or error. status_t err2 = native_window_api_disconnect(aNativeWindow, NATIVE_WINDOW_API_CPU); if (err2 != NO_ERROR) { ALOGE("error pushing blank frames: api_disconnect failed: %s (%d)", strerror(-err2), -err2); if (err == NO_ERROR) { err = err2; } } return err;}
关键步骤如下:
0. 获取我们已经创建的Layer的窗口ANativeWindow
- 断掉之前的BufferQueue连接native_window_api_disconnect,这一步是可选的
- 连接Window到BufferQueue native_window_api_connect
- 设置Buffer的大小尺寸native_window_set_buffers_user_dimensions,可选
- 设置Buffer格式,可选,之前创建Layer的时候已经设置了。
- 设置Buffer的transform
- 处理Buffer的usage,主要的DRM内容的处理
- 设置Buffer的usage,usage由producer的usage和Consumer的usage组成
- 设置scale模式,如果上层给的数据,比如Video,超出Buffer的大小后,怎么处理,是截取一部分还是,缩小。
- 设置permission,设置Buffer,默认true,可选。
- 设置Buffer数量,就是,BufferQueue中有多少个buffer可以用,可选
- 绘制窗口,窗口有一个buffer队列,对每一个buffer都需要绘制。
- dequeueBuffer先拿到一块可用的Buffer,也就是FREE的Buffer。
- Buffer虽然是Free的,但是在异步模式下,Buffer不可能还在使用中,需要等到Fence才能确保buffer没有在被使用
- 往Free的Buffer里面绘制东西,
- 我们这里直接显示全黑,*img = 0
- 将绘制好的Buffer,queue到Buffer队列中,queueBuffer。
- 错误处理,取消掉Buffer,cancelBuffer
- 断开BufferQueue和窗口的连接,native_window_api_disconnect
OK~再编译执行一下,屏幕是不是黑了?
Dumps一下SurfaceFlinger,我们的应用窗口信息如下:
+ Layer 0x7b3ba61000 (NativeBinApp#0) Region transparentRegion (this=0x7b3ba61380, count=1) [ 0, 0, 0, 0] Region visibleRegion (this=0x7b3ba61010, count=1) [ 0, 0, 720, 1280] Region surfaceDamageRegion (this=0x7b3ba61088, count=1) [ 0, 0, -1, -1] layerStack= 0, z=2147483647, pos=(0,0), size=( 720,1280), crop=( 0, 0, -1, -1), finalCrop=( 0, 0, -1, -1), isOpaque=1, invalidate=0, dataspace=Default (0), pixelformat=RGBx_8888 alpha=1.000, flags=0x00000002, tr=[1.00, 0.00][0.00, 1.00] client=0x7b4002d6c0 format= 2, activeBuffer=[ 720x1280: 720, 2], queued-frames=0, mRefreshPending=0 mTexName=2 mCurrentTexture=1 mCurrentCrop=[0,0,0,0] mCurrentTransform=0 mAbandoned=0 - BufferQueue mMaxAcquiredBufferCount=1 mMaxDequeuedBufferCount=1 mDequeueBufferCannotBlock=0 mAsyncMode=0 default-size=[720x1280] default-format=2 transform-hint=00 frame-counter=3 FIFO(0): Slots: [01:0x0] state=FREE [02:0x0] state=FREE
对比看一下,和前面的dump信息有什么不一样?
另外,如果我们将原来的*img = 0替换掉,可以绘制其他一些东西。fillWithCheckerboard可以将屏幕填充为小方块。
fillWithCheckerboard(img, windowSurface->width(), windowSurface->height(), buf->getStride());void fillWithCheckerboard(uint8_t* img, int width, int height, int stride) { bool change = false; for (int y = 0; y < height; y++) { for (int x = 0; x < width; x++) { uint8_t* pixel = img + (4 * (y*stride + x)); if ( x % 10 == 0) { change = !change; } if(change) { pixel[0] = 255; pixel[1] = 255; pixel[2] = 255; } else { pixel[0] = 0; pixel[1] = 0; pixel[2] = 0; } pixel[3] = 0; } if ( y % 10 == 0) { change = !change; } }}
绘制应用,我们这里直接用的API,这些API是怎么工作的,数据怎么送给显示的?接下里,我们将具体分析。
SurfaceFlinger创建Layer
上一章讲到,应用创建Layer时,流程只跟到SurfaceFlinger,SurfaceFlinger是怎么窗口Layer的,和 Layer和BufferQueue又是怎么关联的,我们接着就来看看。
Layer分两种类型:
- bNormal Layer,普通Layer,由createBufferLayer创建,由BufferLayer类描述。
- Coler Layer,由createColorLayer创建,由ColorLayer类描述。
Layer相关类的关系如下:
- BufferLayer和ColorLayer继承Layer类
- Layer类,有LayerBE的一个实例
- BufferLayer实现ContentsChangedListener和FrameAvailableListener两个接口类,主要是监听显示内容的改变。
ColorLayer比较 简单,我们先来看BufferLayer。reateBufferLayer实现如下:
status_t SurfaceFlinger::createBufferLayer(const sp& client, const String8& name, uint32_t w, uint32_t h, uint32_t flags, PixelFormat& format, sp* handle, sp* gbp, sp* outLayer){ ... ... sp layer = new BufferLayer(this, client, name, w, h, flags); status_t err = layer->setBuffers(w, h, format, flags); if (err == NO_ERROR) { *handle = layer->getHandle(); *gbp = layer->getProducer(); *outLayer = layer; } ALOGE_IF(err, "createBufferLayer() failed (%s)", strerror(-err)); return err;}
createBufferLayer时,创建一个BufferLayer。
BufferLayer的构造函数如下:
BufferLayer::BufferLayer(SurfaceFlinger* flinger, const sp& client, const String8& name, uint32_t w, uint32_t h, uint32_t flags) : Layer(flinger, client, name, w, h, flags), mConsumer(nullptr), mTextureName(UINT32_MAX), mFormat(PIXEL_FORMAT_NONE), mCurrentScalingMode(NATIVE_WINDOW_SCALING_MODE_FREEZE), mBufferLatched(false), mPreviousFrameNumber(0), mUpdateTexImageFailed(false), mRefreshPending(false) { ALOGV("Creating Layer %s", name.string()); mFlinger->getRenderEngine().genTextures(1, &mTextureName); mTexture.init(Texture::TEXTURE_EXTERNAL, mTextureName); if (flags & ISurfaceComposerClient::eNonPremultiplied) mPremultipliedAlpha = false; mCurrentState.requested = mCurrentState.active; // drawing state & current state are identical mDrawingState = mCurrentState;}
在LayerBuffer的构造函数中,主要是初始化了一个mTextureName,已经一些状态的初始化;以及调用Layer的构造函数。
Layer::Layer(SurfaceFlinger* flinger, const sp& client, const String8& name, uint32_t w, uint32_t h, uint32_t flags) : contentDirty(false), sequence(uint32_t(android_atomic_inc(&sSequence))), mFlinger(flinger), mPremultipliedAlpha(true), mName(name), mTransactionFlags(0), mPendingStateMutex(), mPendingStates(), mQueuedFrames(0), mSidebandStreamChanged(false), mActiveBufferSlot(BufferQueue::INVALID_BUFFER_SLOT), mCurrentTransform(0), mOverrideScalingMode(-1), mCurrentOpacity(true), mCurrentFrameNumber(0), mFrameLatencyNeeded(false), mFiltering(false), mNeedsFiltering(false), mProtectedByApp(false), mClientRef(client), mPotentialCursor(false), mQueueItemLock(), mQueueItemCondition(), mQueueItems(), mLastFrameNumberReceived(0), mAutoRefresh(false), mFreezeGeometryUpdates(false) { mCurrentCrop.makeInvalid(); uint32_t layerFlags = 0; if (flags & ISurfaceComposerClient::eHidden) layerFlags |= layer_state_t::eLayerHidden; if (flags & ISurfaceComposerClient::eOpaque) layerFlags |= layer_state_t::eLayerOpaque; if (flags & ISurfaceComposerClient::eSecure) layerFlags |= layer_state_t::eLayerSecure; mName = name; mTransactionName = String8("TX - ") + mName; mCurrentState.active.w = w; ... ... init mCurrentState mCurrentState.type = 0; // drawing state & current state are identical mDrawingState = mCurrentState; const auto& hwc = flinger->getHwComposer(); const auto& activeConfig = hwc.getActiveConfig(HWC_DISPLAY_PRIMARY); nsecs_t displayPeriod = activeConfig->getVsyncPeriod(); mFrameTracker.setDisplayRefreshPeriod(displayPeriod); CompositorTiming compositorTiming; flinger->getCompositorTiming(&compositorTiming); mFrameEventHistory.initializeCompositorTiming(compositorTiming);}
Layerd的构造函数中,主要做一些变量的初始化,以及mCurrentState的初始化。
BufferLayer和Layer都是继承RefBase的,还要一个地方做初始化,那就是onFirstRef。
Layer的onFirstRef是空的:
void Layer::onFirstRef() {}
BufferLayer的onFirstRef则做了很多操作。在这里我们就看到Producer和Consumer出场了。
void BufferLayer::onFirstRef() { // Creates a custom BufferQueue for SurfaceFlingerConsumer to use sp producer; sp consumer; BufferQueue::createBufferQueue(&producer, &consumer, true); mProducer = new MonitoredProducer(producer, mFlinger, this); mConsumer = new BufferLayerConsumer(consumer, mFlinger->getRenderEngine(), mTextureName, this); mConsumer->setConsumerUsageBits(getEffectiveUsage(0)); mConsumer->setContentsChangedListener(this); mConsumer->setName(mName); if (mFlinger->isLayerTripleBufferingDisabled()) { mProducer->setMaxDequeuedBufferCount(2); } const sp hw(mFlinger->getDefaultDisplayDevice()); updateTransformHint(hw);}
BufferLayer,通过BufferQueue的createBufferQueue,创建了一个buffer队列,一个buffer队列,有一个生产者producer,和一个消费者consumer。
createBufferQueue实现如下:
void BufferQueue::createBufferQueue(sp* outProducer,
sp* outConsumer,
bool consumerIsSurfaceFlinger) {
… …
sp core(new BufferQueueCore());LOG_ALWAYS_FATAL_IF(core == NULL, "BufferQueue: failed to create BufferQueueCore");sp producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger));LOG_ALWAYS_FATAL_IF(producer == NULL, "BufferQueue: failed to create BufferQueueProducer");sp consumer(new BufferQueueConsumer(core));LOG_ALWAYS_FATAL_IF(consumer == NULL, "BufferQueue: failed to create BufferQueueConsumer");*outProducer = producer;*outConsumer = consumer;
}
- 首先创建了一个BufferQueueCore,这个是BufferQueue的核心。
- 然后创建了一个BufferQueueProducer和一个BufferQueueConsumer,注意Producer和Consumer都持有BufferQueueCore的引用。
BufferQueue创建完后,BufferLayer,又对BufferQueueCore中的Producer和Consume进行封装。分别创建了MonitoredProducer和BufferLayerConsumer。
最后,再对创建的mConsumer和mProducer进行初始化。
mConsumer这边主要有:
- setConsumerUsageBits,设置Consumer的usage
- setContentsChangedListener,这种内容改变的监听,注意这里传的是this指针,因为BufferLayer实现了两个接口,还记得不?
- setName,设置Consumer 名
mProducer这边主要有
- setMaxDequeuedBufferCount
根据系统的属性,设置Producer最多可以申请多少个Buffer,默认是3个;如果配置了属性ro.sf.disable_triple_buffer
为true,那就只能用2个。
这个是在SurfaceFlinger初始化时,在SurfaceFlinger的构造函数中决定的。
property_get("ro.sf.disable_triple_buffer", value, "1"); mLayerTripleBufferingDisabled = atoi(value); ALOGI_IF(mLayerTripleBufferingDisabled, "Disabling Triple Buffering");
我们来看看Layer和BufferQueue之间的关系~
解释一下:
- 一个Layer对应一个BufferQueue,一个BufferQueue中有多个Buffer,一般是2个或者3个。
- 一个Layer有一个Producer,一个Consumer
- 结合前面的分析,一个Surface和一个Layer也是一一对应的,和窗口也是一一对应的。
可见,BufferQueue就是两个连接纽带,连接着Producer和Consumer。接下来,我们就来分别看一下Producer和Consumer。
MonitoredProducer是对BufferQueueProducer的封装,其目的,就是Producer销毁时,能通知SurfaceFlinger。这就是取名Monitored的愿意。余下的,MonitoredProducer的很多接口都是直接调,对应的BufferQueueProducer的实现。
销毁监听,就是在MonitoredProducer析构函数中,post一个消息到SurfaceFlinger的主线程中。通知SurFaceFlinger Producer已经销毁,SurfaceFlinger 会将销毁的Producer从mGraphicBufferProducerList中删掉。代码如下:
MonitoredProducer::~MonitoredProducer() { // Remove ourselves from SurfaceFlinger's list. We do this asynchronously // because we don't know where this destructor is called from. It could be // called with the mStateLock held, leading to a dead-lock (it actually // happens). class MessageCleanUpList : public MessageBase { public: MessageCleanUpList(const sp& flinger, const wp& producer) : mFlinger(flinger), mProducer(producer) {} virtual ~MessageCleanUpList() {} virtual bool handler() { Mutex::Autolock _l(mFlinger->mStateLock); mFlinger->mGraphicBufferProducerList.remove(mProducer); return true; } private: sp mFlinger; wp mProducer; }; mFlinger->postMessageAsync(new MessageCleanUpList(mFlinger, asBinder(mProducer)));}
BufferQueueProducer就是Producer真是实现的地方了。前面我们的应用代码中,要绘制一个窗口,有很多个步骤,而每一步的实现,基本都在BufferQueueProducer中。
BufferQueueProducer的类图如下:
其中,dequeueBuffer和queueBuffer是两个非常重要的函数。我们的应用中,是不是通过ANativeWindow的dequeueBuffer函数,获取到一个Buffer,再通过ANativeWindow的queueBuffer,送到显示这边的。具体过程我们稍后我讲解。
再来看Consumer,BufferLayerConsumer继承ConsumerBase。BufferLayerConsumer的构造函数中,主要是一些变量的初始化,主要是ConsumerBase的构造函数:
* frameworks/native/libs/gui/ConsumerBase.cppConsumerBase::ConsumerBase(const sp& bufferQueue, bool controlledByApp) : mAbandoned(false), mConsumer(bufferQueue), mPrevFinalReleaseFence(Fence::NO_FENCE) { // Choose a name using the PID and a process-unique ID. mName = String8::format("unnamed-%d-%d", getpid(), createProcessUniqueId()); // Note that we can't create an sp<...>(this) in a ctor that will not keep a // reference once the ctor ends, as that would cause the refcount of 'this' // dropping to 0 at the end of the ctor. Since all we need is a wp<...> // that's what we create. wp listener = static_cast(this); sp proxy = new BufferQueue::ProxyConsumerListener(listener); status_t err = mConsumer->consumerConnect(proxy, controlledByApp); if (err != NO_ERROR) { CB_LOGE("ConsumerBase: error connecting to BufferQueue: %s (%d)", strerror(-err), err); } else { mConsumer->setConsumerName(mName); }}
在ConsumerBase的构造函数中,给BufferQueue设置了监听,这样Consumer和BufferQueue,就算是连上了。
注意这里的Listener。BufferLayer是实现了BufferLayerConsumer的ContentsChangedListener,在BufferLayer的onFirstRef中,这个Listener被设置给了BufferLayerConsumer。
mConsumer->setContentsChangedListener(this);
BufferLayerConsumer的setContentsChangedListener函数如下:
void BufferLayerConsumer::setContentsChangedListener(const wp& listener) { setFrameAvailableListener(listener); Mutex::Autolock lock(mMutex); mContentsChangedListener = listener;}
可见,在setFrameAvailableListener函数中,BufferLayer的Listener实现被赋值给了mFrameAvailableListener。同时调用setFrameAvailableListener
setFrameAvailableListener的实现在父类ConsumerBase中。
void ConsumerBase::setFrameAvailableListener( const wp& listener) { CB_LOGV("setFrameAvailableListener"); Mutex::Autolock lock(mFrameAvailableMutex); mFrameAvailableListener = listener;}
此时,又被赋值给了mFrameAvailableListener,注意,这里的mFrameAvailableListener是BufferLayer中实现的Listener。
ConsumerBase自身实现ConsumerListener,中构造的Listener,通过代理ProxyConsumerListener,在connect时传给了BufferQueueConsumer。
* frameworks/native/libs/gui/BufferQueueConsumer.cppstatus_t BufferQueueConsumer::connect( const sp& consumerListener, bool controlledByApp) { ATRACE_CALL(); if (consumerListener == NULL) { BQ_LOGE("connect: consumerListener may not be NULL"); return BAD_VALUE; } BQ_LOGV("connect: controlledByApp=%s", controlledByApp ? "true" : "false"); Mutex::Autolock lock(mCore->mMutex); if (mCore->mIsAbandoned) { BQ_LOGE("connect: BufferQueue has been abandoned"); return NO_INIT; } mCore->mConsumerListener = consumerListener; mCore->mConsumerControlledByApp = controlledByApp; return NO_ERROR;}
看明白了吧?BufferLayer实现的ContentsChangedListener被保存在ConsumerBase中mFrameAvailableListener。而ConsumerBase实现的ConsumerListener,被传到BufferQueueConsumer,保存在BufferQueueCore的mConsumerListener中。
所以,Listener的通知路线应该是这样的~
- Producer生产完后,会通过BufferQueueCore中的mConsumerListener通知ConsumerBase
- ConsumerBase,接受到BufferQueueConsumer的通知,再通过BufferLayer传下来的信使mFrameAvailableListener,通知BufferLayer。
- BufferLayer接受到通知后,就可以去消费生产完的Buffer了。
到此,Consumer这边准备就绪了,就等着Producer去生产了。注意一点,在分析应用创建Layer时,会得到一个IGraphicBufferProducer,这个就是对应BufferLayer
sp SurfaceComposerClient::createSurface( ... ... sp gbp; if (parent != nullptr) { parentHandle = parent->getHandle(); } status_t err = mClient->createSurface(name, w, h, format, flags, parentHandle, windowType, ownerUid, &handle, &gbp); ALOGE_IF(err, "SurfaceComposerClient::createSurface error %s", strerror(-err)); if (err == NO_ERROR) { sur = new SurfaceControl(this, handle, gbp); } } return sur;}
让我们回到我们的应用代码~
Native窗口
在应用代码中,我们已经用到几个关键的类,Surface和SurfaceControl,ANativeWindow和ANativeWindowBuffer;他们又是什么的关系呢,怎么和BufferQueue产生联系的呢?
ANativeWindow
ANativeWindow是Native对一个窗口的描述,和Surface是对等的,Why?可以通过接口ANativeWindow_fromSurface()将Surface转换为ANativeWindow。而事实也ANativeWindow是对BufferQueue的Producer端进行一个封装。
ANativeWindow的定义如下,英文的注释很详细
* frameworks/native/libs/nativewindow/include/system/window.hstruct ANativeWindow{#ifdef __cplusplus ANativeWindow() : flags(0), minSwapInterval(0), maxSwapInterval(0), xdpi(0), ydpi(0) { common.magic = ANDROID_NATIVE_WINDOW_MAGIC; common.version = sizeof(ANativeWindow); memset(common.reserved, 0, sizeof(common.reserved)); } /* Implement the methods that sp expects so that it can be used to automatically refcount ANativeWindow's. */ void incStrong(const void* /*id*/) const { common.incRef(const_cast(&common)); } void decStrong(const void* /*id*/) const { common.decRef(const_cast(&common)); }#endif // 相当于从android_native_base_t继承 struct android_native_base_t common; /* flags describing some attributes of this surface or its updater */ const uint32_t flags; /* min swap interval supported by this updated */ const int minSwapInterval; /* max swap interval supported by this updated */ const int maxSwapInterval; /* horizontal and vertical resolution in DPI */ const float xdpi; const float ydpi; /* Some storage reserved for the OEM's driver. */ intptr_t oem[4]; // 设置swap的间隔,也就是设置Producer是同步还是异步 int (*setSwapInterval)(struct ANativeWindow* window, int interval); // dequeue一块buffer,执行后,buffer就不是locked状态,内容不能修改 // 这里会造成block,引起ANR等如果没有空闲Buffer // 这个方法现象不建议使用,现在直接使用下面的dequeueBuffer方法 int (*dequeueBuffer_DEPRECATED)(struct ANativeWindow* window, struct ANativeWindowBuffer** buffer); // 在修改Buffer的内容前,先锁住这个Buffer int (*lockBuffer_DEPRECATED)(struct ANativeWindow* window, struct ANativeWindowBuffer* buffer); // 修改完后,通过此方法将buffer送输出,这个Buffer也没有在用了。 int (*queueBuffer_DEPRECATED)(struct ANativeWindow* window, struct ANativeWindowBuffer* buffer); // 获取我们需要的值 int (*query)(const struct ANativeWindow* window, int what, int* value); // 执行对应的操纵 int (*perform)(struct ANativeWindow* window, int operation, ... ); // 取消掉一个已经被deueue出来的值 int (*cancelBuffer_DEPRECATED)(struct ANativeWindow* window, struct ANativeWindowBuffer* buffer); // dequeueBuffer_DEPRECATED的新版本,使用者自己处理Fence int (*dequeueBuffer)(struct ANativeWindow* window, struct ANativeWindowBuffer** buffer, int* fenceFd); // queueBuffer_DEPRECATED的新版本 int (*queueBuffer)(struct ANativeWindow* window, struct ANativeWindowBuffer* buffer, int fenceFd); // cancelBuffer_DEPRECATED的新版本,必须要和dequeue在同一个线程中 int (*cancelBuffer)(struct ANativeWindow* window, struct ANativeWindowBuffer* buffer, int fenceFd);};
此外,window.h头文件中还提供了很多类型native_window_**的API,这些API就是通过ANativeWindow的perform函数调下去的。API很多,这里就不一一介绍了,前面我们的应用代码中已经使用了不少。
为什么说ANativeWindow和Surface是对等的?我们来看看Surface
Surface
Surface也是BufferQueue在Producer端的封装,每个窗口都有且只有一个自己的Surface(同一时刻)。为什么说ANativeWindow和Surface是对等的,实际上Surface继承ANativeWindow。
* frameworks/native/libs/gui/include/gui/Surface.hclass Surface : public ANativeObjectBase
ANativeWindow是一个模板类,主要是将类似ANativeWindow这样的类型,转换为引用计数控制的类型,实现对象的自动释放。
template class ANativeObjectBase : public NATIVE_TYPE, public REF{
Surface的代码比较多,这里就不贴代码了。但是整体而言,主要如下:
- ANativeWindow的hooks函数,命名为hook_***,总共10个hook函数,如hook_perform,hook_dequeueBuffer等
- window.h头文件中定义的API的分发,命名为dispatch***,总共29个,如dispatchConnect,dispatchSetCrop等
- Surface对hook函数和dispatch函数的具体实现,这些给函数就和BufferQueue交互。
- 窗口,Buffer的很多描述的属性定义在Surface中。
Surface的实现在:
* frameworks/native/libs/gui/Surface.cpp
在构造函数中,主要是变量的初始化,和ANativeWindow的函数的初始化,将hook函数直接赋值给ANativeWindow对应的函数。
根据我们应用的代码,我们来看看具有代表行的一两个流程,就看native_window_set_buffers_format。
* frameworks/native/libs/nativewindow/include/system/window.hstatic inline int native_window_set_buffers_format( struct ANativeWindow* window, int format){ return window->perform(window, NATIVE_WINDOW_SET_BUFFERS_FORMAT, format);}
native_window_api_connect 调的是 ANativeWindow 的 perform 函数,而perform的类型为 NATIVE_WINDOW_SET_BUFFERS_FORMAT。
perform函数是Surface中实现的:
* frameworks/native/libs/gui/Surface.cppint Surface::perform(int operation, va_list args){ int res = NO_ERROR; switch (operation) { ... ... case NATIVE_WINDOW_SET_BUFFERS_FORMAT: res = dispatchSetBuffersFormat(args); break; ... ...}
dispatch函数为dispatchSetBuffersFormat
int Surface::dispatchSetBuffersFormat(va_list args) { PixelFormat format = va_arg(args, PixelFormat); return setBuffersFormat(format);}
Surface的实现为:
int Surface::setBuffersFormat(PixelFormat format){ ALOGV("Surface::setBuffersFormat"); Mutex::Autolock lock(mMutex); if (format != mReqFormat) { mSharedBufferSlot = BufferItem::INVALID_BUFFER_SLOT; } mReqFormat = format; return NO_ERROR;}
设置的Buffer格式被赋值给了mReqFormat。
以此类推,window.h 头文件中的API,都会设置一个类型,然后通过perform函数,调到Surface中的具体实现。
hook的函数也是类似的,我们以ANativeWindow的dequeueBuffer为例,ANativeWindow的dequeueBuffer函数,直接被赋值为Surface的dequeueBuffer。
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) { ... ... status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight, reqFormat, reqUsage, &mBufferAge, enableFrameTimestamps ? &frameTimestamps : nullptr); ... ... sp& gbuf(mSlots[buf].buffer); ... ... *buffer = gbuf.get(); ... ... return OK;}
dequeueBuffer的时候,通过mGraphicBufferProducer的dequeueBuffer,去找到可用Buffer的id,然后根据id去队列里面取Buffer。
这下明白,为什么说 ANativeWindow和Surface是对等的了吧。但是… …
但是,对不对等,取决于是否真是的用到Surface。比如,我不想用 Surface的这个流程,我自己写一个MySurface,继承与ANativeWindow,然后我用自己的MySurface。此时,元芳,你怎么看?
那么ANativeWindow和Surface怎么对等的呢?我们且来看SurfaceControl。
SurfaceControl
SurfaceControl,简单理解就是控制Surface的。怎么控制?我们先来看,什么时候创建的SurfaceControl。
创建Layer的时候,通过createSurface创建了Layer,
sp SurfaceComposerClient::createSurface( ... ... sp gbp; if (parent != nullptr) { parentHandle = parent->getHandle(); } status_t err = mClient->createSurface(name, w, h, format, flags, parentHandle, windowType, ownerUid, &handle, &gbp); ALOGE_IF(err, "SurfaceComposerClient::createSurface error %s", strerror(-err)); if (err == NO_ERROR) { sur = new SurfaceControl(this, handle, gbp); } } return sur;}
创建了Layer后,获取到Layer的handle和BufferQueue的Producer,SurfaceControl中就有了Layer的handle和Producer了。
SurfaceControl的类图:
构造函数如下:
* frameworks/native/libs/gui/SurfaceControl.cppSurfaceControl::SurfaceControl( const sp& client, const sp& handle, const sp& gbp) : mClient(client), mHandle(handle), mGraphicBufferProducer(gbp){}
这里SurfaceControl就和Layer,BufferQueue建立联系了。
再回到我们的代码:
ANativeWindow* aNativeWindow = surfaceControl->getSurface().get();
这里SurfaceControl的getSurface是一个sp,这里是多态的用法,这就是为什么说ANativeWindow和Surface对等了。
getSurface函数如下:
sp SurfaceControl::getSurface() const{ Mutex::Autolock _l(mLock); if (mSurfaceData == 0) { return generateSurfaceLocked(); } return mSurfaceData;}
generateSurfaceLocked函数中,创建一个Surface
sp SurfaceControl::generateSurfaceLocked() const{ // This surface is always consumed by SurfaceFlinger, so the // producerControlledByApp value doesn't matter; using false. mSurfaceData = new Surface(mGraphicBufferProducer, false); return mSurfaceData;}
看到了吧,Surface中的mGraphicBufferProducer是从哪儿来的了吧。在Layer端为MonitoredProducer,Surface这边是Binder的Bp端。
我们先来看Surface相关类的关系吧
看了Surface相关的关系类图,再和SurfaceFlinger,Layer相关的关系类似结合,应用和SurfaceFlinger服务的关系是不是就很清楚了。
到此,应用该做的准备工作都准备完了,应用端主要通过IGraphicBufferProducer和ISurfaceComposerClient两个接口SurfaceFlinger进行交互。
在开始下面的知识之前,我们先来看看这个LayerCleaner
窗口销毁的善后处理
应用被销毁后,Client端就被清理了,SurfaceControl,SurfaceComposerClient,被销毁。但是服务端,SurfaceFlinger是另外一个进程,为应用进程申请的相关资源什么很好释放呢?
关键还是看上面类图中的Handler。我们就来看一下流程:
SurfaceControl::~SurfaceControl(){ destroy();}
在destroy函数中,销毁应用进程中的资源:
void SurfaceControl::destroy(){ if (isValid()) { mClient->destroySurface(mHandle); } // clear all references and trigger an IPC now, to make sure things // happen without delay, since these resources are quite heavy. mClient.clear(); mHandle.clear(); mGraphicBufferProducer.clear(); IPCThreadState::self()->flushCommands();}
而服务端的,有两种方式:
- 直接通过 Client destroySurface:
* frameworks/native/services/surfaceflinger/Client.cppstatus_t Client::destroySurface(const sp& handle) { return mFlinger->onLayerRemoved(this, handle);}
status_t SurfaceFlinger::onLayerRemoved(const sp& client, const sp& handle){ // called by a client when it wants to remove a Layer status_t err = NO_ERROR; sp l(client->getLayerUser(handle)); if (l != NULL) { mInterceptor.saveSurfaceDeletion(l); err = removeLayer(l); ALOGE_IF(err<0 && err != NAME_NOT_FOUND, "error removing layer=%p (%s)", l.get(), strerror(-err)); } return err;}
但是,注意这里的isValid()
如果isValid无效呢?
这个时候,我们就要通过mClient和mHandle。这个时候是引用计数控制的自动释放。
- 引用计数控制自动释放
mClient.clear(); mHandle.clear();
clear函数会是否对象的应用,最终调用析构函数:
Client::~Client(){ const size_t count = mLayers.size(); for (size_t i=0 ; i l = mLayers.valueAt(i).promote(); if (l != nullptr) { mFlinger->removeLayer(l); } }}
这里是不是和destroySurface函数是异曲同工之处。
再来看Handle:
* frameworks/native/services/surfaceflinger/Layer.h class Handle : public BBinder, public LayerCleaner { public: Handle(const sp& flinger, const sp& layer) : LayerCleaner(flinger, layer), owner(layer) {} wp owner; };
Handle析构时,会调父类的析构:
protected: ~LayerCleaner() { // destroy client resources mFlinger->onLayerDestroyed(mLayer); } };
LayerCleaner的析构中同样调的SurfaceFlinger的onLayerRemoved函数。再调的removeLayer
status_t SurfaceFlinger::removeLayer(const sp& layer, bool topLevelOnly) { ... ... const auto& p = layer->getParent(); ssize_t index; if (p != nullptr) { ... ... index = p->removeChild(layer); } else { index = mCurrentState.layersSortedByZ.remove(layer); } ... ... layer->onRemovedFromCurrentState(); mLayersPendingRemoval.add(layer); mLayersRemoved = true; mNumLayers -= 1 + layer->getChildrenCount(); setTransactionFlags(eTransactionNeeded); return NO_ERROR;}
删除Layer时,主要做了以下几件事:
- 将Layer从父Layer中删掉,或者从mCurrentState中删掉,放到待删除Layer列表中
- onRemovedFromCurrentState,清理Layer,如果是父Layer,子Layer也删掉
- setTransactionFlags,通知SurfaceFlinger更新,更新后,我们删掉的Layer就没有了,屏幕就不显示了。
最后销毁Layer
* frameworks/native/services/surfaceflinger/Layer.cppLayer::~Layer() { sp c(mClientRef.promote()); if (c != 0) { c->detachLayer(this); } for (auto& point : mRemoteSyncPoints) { point->setTransactionApplied(); } for (auto& point : mLocalSyncPoints) { point->setFrameAvailable(); } mFrameTracker.logAndResetStats(mName);}
善终… …
更多相关文章
- android中自定义appWidget
- Android应用程序访问硬件驱动(JNI方式)
- 自定义 Android(安卓)Preference——SpinnerPreference的私人定
- android 创建自己的TabActivity
- Android(安卓)Sensor框架简述(一)
- Android(安卓)技术专题系列之二 -- telephony(转载)
- Unable to load script.Make sure you're either running a metr
- Android(安卓)JNI/Hardware 加载(二)
- Android(安卓)C++中调用java