How To Draw Objects On Textureview Camera Stream Preview And Record The Stream With Objects?

I need help with an application I am working on. The application has to have a custom Camera interface to record a video with audio and have to add some objects in realtime on the

Solution 1:

Modifying a video in real time is a high processor and hence, high battery overhead operation - I am sure you know this but its worth saying that if you can add your modifications on the server side, maybe by sending the stream along with a timestamped set of text overlays to the server, you should have more horsepower server side.

The following code will add text and an image to a still picture or frame captured by Camera2 on Android. I have not used it with video so can't comment on speed and whether it is practical to do this with a real time video stream - it wasn't optimised for this but it should be a starting point for you:

//Copy the image to a BitBufferByteBuffer buffer = mCameraImage.getPlanes()[0].getBuffer();
        byte[] bytes = new byte[buffer.remaining()];
        Log.d(TAG,"ImageSaver bytes.length: " + bytes.length);
        buffer.get(bytes);
        BitmapFactory.Options opt = newBitmapFactory.Options();
        opt.inMutable = true;
        Bitmap cameraBitmap = BitmapFactory.decodeByteArray(bytes,0,bytes.length, opt);
        if (cameraBitmap == null) {
            Log.d(TAG,"ImageSaver cameraBitmap is null");
            return;
        } else {
            camImgBitmap = cameraBitmap;
        }

        //Modify captured picture by drawing on canvasCanvas camImgCanvas = newCanvas(camImgBitmap);

        //Draw an image in the middleDrawable d = ContextCompat.getDrawable(this, R.drawable.image_to_add);
        int bitMapWidthCenter = camImgBitmap.getWidth()/2;
        int bitMapheightCenter = camImgBitmap.getHeight()/2;
        int imageToDrawSize = camImgBitmap.getWidth()/10;
        int rTop = bitMapheightCenter - sightsSize;
        int rLeft = bitMapWidthCenter - sightsSize;
        int rRight = bitMapWidthCenter + sightsSize;
        int rBot = bitMapheightCenter + sightsSize;
        d.setBounds(rLeft, rTop, rRight, rBot);
        d.draw(camImgCanvas);

        //Now Draw in some textPaint paint = newPaint();
        paint.setColor(Color.GREEN);
        int textSize = camImgBitmap.getHeight()/20;
        int textPadding = 40;
        paint.setTextSize(textSize);
        camImgCanvas.drawText("Name: " + text1, textPadding, (camImgBitmap.getHeight() - (textSize * 2) ) - textPadding, paint);
        camImgCanvas.drawText("Time: " + text2 + " degrees", textPadding, (camImgBitmap.getHeight() - textSize) - textPadding, paint);

Solution 2:

Likely the most performant option is to pipe the camera feed straight into the GPU, draw on top of it there, and from there render to the display and a video encoder directly.

This is what many video chat apps do, for example, for any effects.

You can use a SurfaceTexture to connect camera2 to EGL, and then you can render the preview onto a quad, and then your additions on top.

Then you can render to a screen buffer (GLSurfaceView for example), and to a separate EGLImage from a MediaRecorder/MediaCodec Surface.

There's a lot of code involved there, and a lot of scaffolding for EGL setup, so it's hard to point to any simple examples.