Renderscript Documentation and Advice - Android

Question

I have been following this guide on how to use Render-script on Android.

http://www.jayway.com/2014/02/11/renderscript-on-android-basics/

My code is this (I got a wrapper class for the script):

public class PixelCalcScriptWrapper {

    private Allocation inAllocation;
    private Allocation outAllocation;
    RenderScript rs;
    ScriptC_pixelsCalc script;

    public PixelCalcScriptWrapper(Context context){
        rs = RenderScript.create(context);
        script = new ScriptC_pixelsCalc(rs, context.getResources(), R.raw.pixelscalc);
    };

    public void setInAllocation(Bitmap bmp){
        inAllocation = Allocation.createFromBitmap(rs,bmp);
    };

    public void setOutAllocation(Bitmap bmp){
        outAllocation = Allocation.createFromBitmap(rs,bmp);
    };

    public void forEach_root(){
        script.forEach_root(inAllocation, outAllocation);
    }
}

This methods calls the script:

public Bitmap processBmp(Bitmap bmp, Bitmap bmpCopy) {

    pixelCalcScriptWrapper.setInAllocation(bmp);
    pixelCalcScriptWrapper.setOutAllocation(bmpCopy);
    pixelCalcScriptWrapper.forEach_root();

    return bmpCopy;
};

and here is my script:

#pragma version(1)
#pragma rs java_package_name(test.foo)

void root(const uchar4 *in, uchar4 *out, uint32_t x, uint32_t y) {
    float3 pixel = convert_float4(in[0]).rgb;

    if(pixel.z < 128) {
        pixel.z = 0;
    }else{
        pixel.z = 255;
    }
    if(pixel.y < 128) {
        pixel.y = 0;
    }else{
        pixel.y = 255;
    }
    if(pixel.x < 128) {
        pixel.x = 0;
    }else{
        pixel.x = 255;
    }

    out->xyz = convert_uchar3(pixel);
}

Now where can I find some documentation about this ?

For example, I have these questions:

1) What does this convert_float4(in[0]) do ?

2) What does the rgb return here convert_float4(in[0]).rgb;?

3) What is float3 ?

4) I don't know where to start with this line out->xyz = convert_uchar3(pixel);

5) I am assuming in the parameters, in and out are the Allocations passed? what are x and y?

Answer 1

1) In the kernel, the in pointer is a 4-element unsigned char, that is, it represents a pixel color with R, G, B and A values in the 0-255 range. So convert_float4 simply casts each of the four uchar as a float. In this particular code you are using, it probably doesn't make much sense to work with floats, since you're doing a simple threshold, and you could just as well had worked with the uchar data directly. Using floats is better aimed when doing other types of image processing algorithms where you do need to have the extra precision (example: blurring an image).

2) The .rgb suffix is a shorthand to return only the first three values of the float4, i.e. the R, G, and B values. If you had used only .r it would give you the first value as a regular float, if you had used .g it would give you the second value as a float, etc... These three values are then assigned to that float3 variable, which now represents the pixel with only three color channels (that is, no A alpha channel).

3) See #2.

4) Now convert_uchar3 is again another cast that converts the float3 pixel variable back to a uchar3 variable. You are assigning the three values to each of the x, y, and z elements in that order. This is probably a good time to mention that X, Y and Z are completely interchangeable with R, G and B. That statement could just as well have used out->rgb, and it would actually have been more readable that way. Note that out is a uchar4, and by doing this, you are assigning only the first three "rgb" or "xyz" elements in that pointer, the fourth element is left undefined here.

5) Yes, in is the input pixel, out is the output pixel. Then x and y are the x, and y coordinates of the pixel in the overall image. This kernel function is going to be called once for every pixel in the image/allocation you're working with, and so it's usually good to know what coordinate you're at when processing an image. In this particular example since it's only thresholding all pixels in the same way, the coordinates are irrelevant.

Good documentation on RenderScript is very hard to find. I would greatly recommend you take a look at these two videos though, as they will give you a much better sense of how RenderScript works:

AnDevCon: A Deep Dive into RenderScript

Google I/O 2013 - High Performance Applications with RenderScript

Keep in mind that both videos are a couple years old, so some minor details may have changed on the recent APIs, but overall, those are probably the best sources of information for RS.

Answer 2

http://developer.android.com/guide/topics/renderscript/reference/rs_convert.html#android_rs:convert

What does this convert_float4(in[0]) do? convert_float4 will convert from a uchar4 to a float4; .rgb turns it into a float3 of the first 3 elements.

What does the rgb return? RenderScript vector types have .r .g .b .a or .x .y .z .w representing the first, second, third and forth element respectively. You can use any combination (e.g. .xy or .xwy)

What is float3? float3 is a "vector type" sort of like a float but 3 of them. There are float2, float3 and float4 vector types of float. (there are uchar4, int4 etc.)

http://developer.android.com/guide/topics/renderscript/reference/overview.html might be helpful

I hope this helps.