Why "current_thread" identifier is not in "_current_frames" dictionary?

34 views Asked by At

I am writing a stack profiler for python using Rust. Here is the profiler code to collect stack trace:

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
use pyo3::types::{PyAny, PyDict};
use pyo3::wrap_pyfunction;
use std::thread;
use std::time::Duration;
use pyo3::types::PyString;
use pyo3::types::PyTuple;
use pyo3::types::PyList;

/// Function to start the profiler
#[pyfunction]
fn start_profiler(py: Python, interval: u64) -> PyResult<()> {
    thread::spawn(move || -> PyResult<()> {
        loop {
            thread::sleep(Duration::from_millis(interval));
            let gil_guard = Python::acquire_gil();
            let py = gil_guard.python();
            println!("Profiler thread running");

            // Collect stack trace of the thread holding the GIL
            if let Ok(sys) = py.import("sys") {
                if let Ok(frames) = sys.call1("_current_frames", ()) {
                    if let Ok(frame_dict) = frames.downcast::<PyDict>() {
                        println!("Active Python thread idents:");
                        for (key, _value) in frame_dict.iter() {
                            println!("{:?}", key);
                        }
                        if let Ok(threading) = py.import("threading") {
                            if let Ok(current_thread) = threading.call_method0("current_thread") {
                                if let Ok(ident) = current_thread.getattr("ident").and_then(|i| i.extract::<u64>()) {
                                    match frame_dict.get_item(ident) {
                                        Some(frame) => {
                                            if let Ok(traceback) = py.import("traceback") {
                                                if let Ok(trace) = traceback.call_method1("format_stack", (frame,)) {
                                                    println!("Stack trace of Python thread {}: {:?}", ident, trace);
                                                } else {
                                                    println!("Failed to format stack trace for thread ident {}", ident);
                                                }
                                            }
                                        },
                                        None => println!("No frame found for the given ident: {}", ident),
                                    }
                                } else {
                                    println!("Failed to extract ident");
                                }
                            } else {
                                println!("Failed to get current thread");
                            }
                        } else {
                            println!("Failed to import threading");
                        }
                    } else {
                        println!("Failed to downcast to PyDict");
                    }
                } else {
                    println!("Failed to call _current_frames");
                }
            } else {
                println!("Failed to import sys");
            }    
        }
    });
    Ok(())
}

/// A Python module implemented in Rust.
#[pymodule]
fn rust_profiler(_py: Python, m: &PyModule) -> PyResult<()>{
    m.add_function(wrap_pyfunction!(start_profiler, m)?)?;
    Ok(())
}

The intension of the code is to create a background thread to monitor the python application threads. During sampling, I intend to collect the stack trace of the python thread that holds the GIL last time. I am using "current_thread" to find the thread. However, the code cannot find the current thread's identifier in the "_current_frame" dictionary. Following is the output of the profiler. The last line prints the thread id inside the python code.

...
Profiler thread running
Active Python thread idents: 140345858855744
No frame found for the given ident: 140345831622400

Current thread ID: 140345858855744

Can you explain why I cannot monitor the python thread that was holding the GIL? Which current thread has an identifier 140345858855744? Is it the Rust thread that I spawning? How can I fix the profiler?

0

There are 0 answers