Discuss / Python / 按照@三尺蛮腰 的改了,可是master和worker的链接出现问题,我两个都是在本机上跑的

按照@三尺蛮腰 的改了,可是master和worker的链接出现问题,我两个都是在本机上跑的

Topic source
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# task_master.py

import random, time, queue
from multiprocessing.managers import BaseManager
from multiprocessing import freeze_support

# 发送任务的队列:
task_queue = queue.Queue
# 接收结果的队列:
result_queue = queue.Queue

# 从BaseManager继承的QueueManager:
class QueueManager(BaseManager):
    pass
def test():
    # 把两个Queue都注册到网络上, callable参数关联了Queue对象:
    QueueManager.register('get_task_queue', callable=task_queue)
    QueueManager.register('get_result_queue', callable=result_queue)
    # 绑定端口5000, 设置验证码'abc':
    manager = QueueManager(address=('172.24.28.56', 5000), authkey=b'abc')
    # 启动Queue:
    manager.start()
    # 获得通过网络访问的Queue对象:
    task = manager.get_task_queue()
    result = manager.get_result_queue()
    # 放几个任务进去:
    for i in range(10):
        n = random.randint(0, 10000)
        print('Put task %d...' % n)
        task.put(n)
    # 从result队列读取结果:
    print('Try get results...')
    for i in range(10):
        r = result.get(timeout=10)
        print('Result: %s' % r)
    # 关闭:
    manager.shutdown()
    print('master exit.')

if __name__ == '__main__':
    freeze_support()
    test()

worker没改动,结果是这样

master界面:

D:\programlearnning\Python\code>python task_master.py
Put task 4175...
Put task 651...
Put task 65...
Put task 2521...
Put task 151...
Put task 4023...
Put task 6037...
Put task 1923...
Put task 89...
Put task 3667...
Try get results...
Traceback (most recent call last):
  File "task_master.py", line 45, in <module>
    test()
  File "task_master.py", line 37, in test
    r = result.get(timeout=10)
  File "<string>", line 2, in get
  File "C:\Python34\lib\multiprocessing\managers.py", line 747, in _callmethod
    raise convert_to_error(kind, result)
queue.Empty

worker界面:

D:\programlearnning\Python\code>python task_worker.py
Connect to server 172.24.28.56...
task queue is empty.
task queue is empty.
task queue is empty.
task queue is empty.
task queue is empty.
task queue is empty.
Traceback (most recent call last):
  File "task_worker.py", line 30, in <module>
    n = task.get(timeout=1)
  File "<string>", line 2, in get
  File "C:\Python34\lib\multiprocessing\managers.py", line 732, in _callmethod
    kind, result = conn.recv()
  File "C:\Python34\lib\multiprocessing\connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "C:\Python34\lib\multiprocessing\connection.py", line 416, in _recv_bytes
    buf = self._recv(4)
  File "C:\Python34\lib\multiprocessing\connection.py", line 383, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。

我发现我是最后又按照@啊呜说 改了,所以出错了,按照@三尺蛮腰的是对的

redglass2015

#3 Created at ... [Delete] [Delete and Lock User]

我也出现了同样的问题,你是怎么改好的?

baiyuqin

#4 Created at ... [Delete] [Delete and Lock User]

我一开始也是同样的问题,后来又仔细看了博主的文,发现是要开两个cmd,在一个cmd里面运行master的时候,另一个cmd紧接着运行worker就可以了,最后结果会在运行master的cmd里现实,worker只是干活,最后不显示结果的。

Adam常

#5 Created at ... [Delete] [Delete and Lock User]

两个cmd,运行task_worker.py速度要快…………

虚空螺旋

#6 Created at ... [Delete] [Delete and Lock User]

楼上说也对对,我开始用虚拟机模拟两台机器的时候。。手慢了没注意就一直。。ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝,无法连接。

黑岩74032

#7 Created at ... [Delete] [Delete and Lock User]

master里result.get(timeout=10),这里需要在10s获取到信息,否则就抛出Queue.empty,层主明显是master完全结束才执行task的,这样master抛出Queue.empty同时这个5000端口关闭,这样task的connect也就失败了

suzy369

#8 Created at ... [Delete] [Delete and Lock User]

个人感觉 QueueManager.register('get_task_queue',callable=return_task_queue) 这边callable的参数必须是可调用的,如果直接给变量名是不可以的,所以老师在例子中就通过最简单lambda表达式匿名函数的形式。。。。


  • 1

Reply