Write Python with Bing Chat: Coroutines and Threading

Conclusions for the impatient:

Bing Chat's python code generation is good for: 1. learning APIs and code structures for those who are new to the topic. 2. boiler plate code. 3. some useful hints with some smart prompts and interpretation. But it is far from replacing a programmer.

I exercised a challenging python programming topic "coroutine and threading" with Bing Chat, which I believe uses GPT 3.5 by default (somehow I don't see the option of using GPT 4.0 any more at the loading of the chat window).

Me: "write a program that has a main thread and a worker thread. The main thread instantiates 2 coroutines, but those two are scheduled to run on the worker thread. The main thread then waits for the coroutines to finish."

Bing chat:

import asyncio
import threading

async def coroutine1():
    print("Coroutine 1 started")
    await asyncio.sleep(1)
    print("Coroutine 1 finished")

async def coroutine2():
    print("Coroutine 2 started")
    await asyncio.sleep(2)
    print("Coroutine 2 finished")

def worker_thread():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_forever()

# Create the worker thread
thread = threading.Thread(target=worker_thread)
thread.start()

# Run the coroutines on the worker thread
loop = asyncio.get_event_loop()
coro1 = coroutine1()
coro2 = coroutine2()
future1 = asyncio.run_coroutine_threadsafe(coro1, loop)
future2 = asyncio.run_coroutine_threadsafe(coro2, loop)

# Wait for the coroutines to finish
future1.result()
future2.result()

# Stop the worker thread
loop.call_soon_threadsafe(loop.stop)
thread.join()        

Me: "The code actually hang at the line of "future1.result()". "

Bing chat: "future1.result() call blocks forever because it is executed in the main thread, which is also running the event loop. This creates a deadlock situation where the main thread is waiting for the future to complete, but the future can’t complete because the main thread is blocked."

But its subsequent suggestion on the change was not really useful as it rewrites the worker thread to do everything including scheduling coroutines, which was not what I originally asked.

Me: "This is not correct. In my original code, the loop runs in worker thread. Main thread schedules coro1 and coro2 with that loop. Why it is still blocked on future1.result()."

Bing chat: "I apologize for the confusion. You are correct. In your original code, the event loop runs in the worker thread, and the main thread schedules coro1 and coro2 with that loop...."

But its subsequent suggestion was exactly the same as the previous one. Hence no progress.

At this point, I decided to fix the code myself. Bing Chat did provide a useful hint that the main thread is running an event loop. I suspected that the main thread was not getting the same event loop as the one set by the worker thread. I debugged and eventually made two fixes:

1. Blocking the main thread from getting event loop till after worker_thread set it.
2. But the above though necessary, turned out to be not enough. "asyncio.get_event_loop()" actually returns a thread local event loop which would still be different from the one in worker_thread.

The final fix with some debugging code:

import asyncio
import threading

async def coroutine1():
    print(f"Coroutine 1 started in {threading.current_thread().name}")
    await asyncio.sleep(1)
    print("Coroutine 1 finished")

async def coroutine2():
    print(f"Coroutine 2 started in {threading.current_thread().name}")    
    await asyncio.sleep(2)
    print("Coroutine 2 finished")

cond = threading.Condition()
worker_state = "starting"
worker_loop = None
def worker_thread():
    with cond:
        global worker_state, worker_loop
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        print(f"done setting event loop: {id(loop)}")
        worker_state = "running"
        worker_loop = loop
        cond.notify()
    loop.run_forever()

async def main():
    # Create the worker thread
    thread = threading.Thread(target=worker_thread, name="worker")
    thread.start()

    with cond:
        global worker_state
        while worker_state != "running":
            cond.wait()

    # Run the coroutines on the worker thread\
    global worker_loop
    print(f"main thread to get loop: {id(worker_loop)}")
    coro1 = coroutine1()
    coro2 = coroutine2()
    future1 = asyncio.run_coroutine_threadsafe(coro1, worker_loop)
    future2 = asyncio.run_coroutine_threadsafe(coro2, worker_loop)

    print("finished scheduling")
    # Wait for the coroutines to finish
    future1.result()
    future2.result()

    # Stop the worker thread
    worker_loop.call_soon_threadsafe(worker_loop.stop)
    thread.join()

await main()