I have a system where I am able to collect 8-bit Gray images from a camera, place the data into a WriteableBitmap and display the images on a WPF Image object. This work happens in a camera thread. I used this article to help me: How to create a BitmapImage from a pixel byte array (live video display)
What I am trying to to do is make a copy of a subset of the pixel data of the image data. During the frame update in the camera thread I am trying to create the copy of the data in a separate byte array. My code appears to work at first, but then after a few iterations my buffer variable goes from having a range of grey levels (0-255) to only having values of 255 in every array element. The variable appears to accumulate data and max out as opposed to reset each time the background worker is called. I will present my code below.
Can anyone see and describe to me what I am doing wrong? Thank you.
public partial class MainWindow : Window
{
[DllImport("Kernel32.dll",EntryPoint="RtlMoveMemory")]
public static extern void CopyMemory(IntPtr Destination, IntPtr Source,
uint Length);
// Declarations
var pData = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(byte))*FrameSize);
var pFocusData = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(byte))
*FrameSize);
BackgroundWorker bw = new BackgroundWorker();
static CameraWorker cWorker;
static Thread cThread;
WriteableBitmap wbm;
public IntPtr wbmBackBuffer;
const int FrameSize = 1944*2592;
// CameraWorker Event Handler
void CW_FrameUpdated(object sender, CameraWorkerEventArgs e)
{
if (!e.Updated) return;
// e.pData is an IntPtr containing the camera frame data
CopyMemory(this.wbmBackBuffer, e.pData, FrameSize);
this.Dispatcher.Invoke(wbm.Lock);
this.Dispatcher.Invoke(()=>{ wbm.AddDirtyRect(
new Int32Rect(0,0,wbm.PixelWidth,wbm.PixelHeight)); });
this.Dispatcher.Invoke(wbm.Unlock);
// The above works and I get streaming data to my view port.
// Now I want to make a copy of the pixel data to send to another thread
// for processing. This is where I am having trouble.
if (bw.IsBusy) return;
CopyMemory(pFocusData, e.pData, FrameSize);
var args = new List<object>();
args.Add(pFocusData);
bw.RunWorkerAsync(args);
}
// BackgroundWorker event handlers
void bw_DoWork(object sender, DoWorkEventArgs e)
{
// This is where I see the result of the problem when debugging.
List<object> argu = e.Argument as List<object>;
var pData = (IntPtr) argu[0];
var fullFrame = new byte[FrameSize];
Marshal.Copy(pData,fullFrame,0,FrameSize);
// Perform operations on the byte array data.
// I extract a subregion from the byte array to process, however after a
// couple of iterations, all values in fullFrame equal 255. The pData that
// is coming in should be a copy of the pixel data that is being displayed
// on the screen. While the screen keeps updating with a live video image,
// the frameData variable appears to keep accumulating rather than resetting
// with each backgroundworker call.
}
void bw_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
// Update UI elements using Dispatcher with data obtained during DoWork.
}
// Window event handlers
private void MainWindow_Initialized(object sender, EventArgs e)
{
// Set up the WBM
wbm = new WriteableBitmap(width,height,96d,96d,PixelFormats.Gray8,null);
this.wbmBackBuffer = wbm.BackBuffer;
// Set up the camera to grab frames in another thread
cWorker = new CameraWorker(camera);
cWorker.CameraFrameUpdated += CW_FrameUpdated;
cThread = new Thread(new ThreadStart(cWorker.ThreadRun));
cThread.Start();
while(!cThread.IsAlive);
// Set up the background worker for processing image data
bw.DoWork += bw_DoWork;
bw.RunWorkerCompleted += bw_RunWorkerCompleted;
// Bind the image data to the Image object that has the name "viewer"
viewer.Source = wbm;
}
private void MainWindow_Closing(object sender,
System.ComponentModel.CancelEventArgs e)
{
Marshal.FreeHGlobal(pData);
}
}
EDIT: I corrected the typo that Erti-Chris Eelmaa pointed out. It was just a transcription error on my part in showing the relevant portion of my code.
EDIT #2:
1) What happens if you do your BW stuff after if (!e.Updated) return; line? Will the wbm start having this accumulation error and your BW will be fine?
There is no difference in behavior. The wbm is still fine and my BW variable accumulates.
2) BackgroundWorker.IsBusy property will be false after bw_DoWork is finished. Are you okay with this behavior?
My intent was to only process a new frame when the BW was finished. I thought that IsBusy lasts until RunWorkerCompleted is run? I don't need to process every frame.
I tried to simply create a new BW for every call to CW_FrameUpdated, but that did not solve the problem either.
3) What does the MoveMemory do? Does it copy memory from SOURCE to DESTINATION without changing anything in SOURCE? Does it even copy anything?
RtlMoveMemory (CopyMemory) is supposed to copy bytes from one area of memory to the other. I thought that by having allocated two separate spaces of equal size by the AllocHGlobal function I could do a rapid copy of the 8MB of data from one variable to the other. From what I have been reading it appears that I am doing something that the managed memory does not like. I need a fast deep copy of the data. I will try System.Buffer.BlockCopy again in case I missed something the first time.
4) What buffer variable are you speaking about? On every stage, verify ALL BUFFERS and nail down the EXACT place where the buffers differ. You need to create function DumpMemory(IntPtr unmanagedMemory). You can cast IntPtr to byte and use fixed() statement to do so.*
I will set up the DumpMemory function and let you know what I find. In the meantime, here is the data flow and roster of relevant variables.
pData (IntPtr): unmanaged memory containing 1944*2592 bytes of image data in Gray8 format with no header
wbm (WriteableBitmap): WPF variable that is bound to an Image object on the main window
wbmBackBuffer (IntPtr): local variable that points to the same location as the wbm.BackBuffer
pData is copied to wbmBackBuffer using CopyMemory, and because wbm is bound to the Image object, the current image frame is updated on the main window
pFocusData (IntPtr): this is a local pointer that is allocated memory in the size of a full frame of data
pData is copied to pFocusData through CopyMemory.
fullFrame (byte []): this is a byte array copy of pFocusData. This is where I see the accumulation occur.
EDIT #3: I finally solved the issue. It turns out there was a logic error in the sub-arrays I was selecting. My view port is about 800*400 while the image array is about 3.5 times larger. I did not scale the coordinates properly, so my sample regions were all looking at a small and similar region of the frame data. I was able to notice this by following the 4th suggestion and using a memory dump to see exactly what was going on at each step. It helped me see that nothing was wrong with the code ultimately and what I thought was accumulation was actually just the camera saturating in a small region.
The good news is that the code posted above is correct and works. I ultimately ended up using Buffer.BlockCopy and created a duplicate frame that I pass through the camera worker event args.
Thank you very much for your help Erti-Chris Eelmaa!
You're getting tired, it's time for a Christmas now :P
// edit, so looking at your code, I have few questions / suggestions.
1) What happens if you do your BW stuff after
if (!e.Updated) return;line? Will the wbm start having this accumulation error and your BW will be fine?2) BackgroundWorker.IsBusy property will be false after bw_DoWork is finished. Are you okay with this behavior?
3) What does the MoveMemory do? Does it copy memory from SOURCE to DESTINATION without changing anything in SOURCE? Does it even copy anything?
4) What buffer variable are you speaking about? On every stage, verify ALL BUFFERS and nail down the EXACT place where the buffers differ. You need to create function DumpMemory(IntPtr unmanagedMemory). You can cast IntPtr to byte* and use fixed() statement to do so.