Discuss / Python / 请各位同学帮帮我,喜闻乐见的Event loop is closed已经困扰了我好几天。

请各位同学帮帮我,喜闻乐见的Event loop is closed已经困扰了我好几天。

Topic source

被这个Event loop is closed困扰了好几天,求各位指点。

import asyncio
from faker import Faker
import logging
import aiomysql
from .orm import create_pool
from .models import User

logging.basicConfig(level=logging.INFO)
f=Faker()
loop = asyncio.get_event_loop()


@asyncio.coroutine
def test1():  #这里使用廖老师封装的orm,保存User实例
     yield from create_pool(loop,host='192.168.99.2',port=3306,
                            user='pi',password='raspberry@',db='awesome')
    user = User(name=f.first_name(),email=f.email(),passwd=f.state(),image=f.company())
    yield from user.save()


loop.run_until_complete(test1())
loop.close()

这样写会出现Event loop is closed。 它这个Event loop和我自定义的loop没有一分钱的关系吧

INFO:root:found model: User (table:users)
INFO:root: found mapping: id --> <StringField,varcahr(50:None>
INFO:root: found mapping: passwd --> <StringField,varcahr(50):None>
INFO:root: found mapping: admin --> <BooleanField,boolean:None>
INFO:root: found mapping: email --> <StringField,varchar(50):None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: image --> <StringField,varcahr(50):None>
INFO:root: found mapping: name --> <StringField,varchar(50):None>
INFO:root:found model: Blog (table:blogs)
INFO:root: found mapping: id --> <StringField,varchar(50):None>
INFO:root: found mapping: user_image --> <StringField,varchar(500:None>
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: user_name --> <StringField,varchar(50:None>
INFO:root: found mapping: summary --> <StringField,varchar(200:None>
INFO:root: found mapping: name --> <StringField,varchar(50:None>
INFO:root: found mapping: user_id --> <StringField,varchar(50:None>
INFO:root:found model: Comment (table:comments)
INFO:root: found mapping: blog_id --> <StringField,varchar(50):None>
INFO:root: found mapping: id --> <StringField,varchar(50:None>
INFO:root: found mapping: user_image --> <StringField,varchar(500):None>
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: user_id --> <StringField,varchar(50):None>
INFO:root: found mapping: user_name --> <StringField,varchar(50):None>
INFO:root: create databases connection pool...
INFO:root:passwd
INFO:root:admin
INFO:root:email
INFO:root:created_at
INFO:root:image
INFO:root:name
INFO:root:id
INFO:root:SQL: insert into `users` (`passwd`, `admin`, `email`, `created_at`, `image`, `name`,`id`) values (?, ?, ?, ?, ?, ?, ?)
INFO:root:...affected: 1
INFO:root:...............User.save()...................
Exception ignored in: <bound method Connection.__del__ of <aiomysql.connection.Connection object at 0x02A10F10>>
Traceback (most recent call last):
  File "D:\Python35-32\lib\site-packages\aiomysql\connection.py", line 694, in __del__
  File "D:\Python35-32\lib\site-packages\aiomysql\connection.py", line 260, in close
  File "D:\Python35-32\lib\asyncio\selector_events.py", line 573, in close
  File "D:\Python35-32\lib\asyncio\base_events.py", line 497, in call_soon
  File "D:\Python35-32\lib\asyncio\base_events.py", line 506, in _call_soon
  File "D:\Python35-32\lib\asyncio\base_events.py", line 334, in _check_closed
RuntimeError: Event loop is closed

Process finished with exit code 0

::::::::::::::::::::::::::::下面测试第2种写法:::::::::::::::::::::::::::::::::::::::::::

#import 部分和上边相同
@asyncio.coroutine
def test2():  #这里使用aiomysql文档的示例写法,直接执行insert语句方式插入数据
    pool = yield from aiomysql.create_pool(host='192.168.99.2', port=3306,
              user='pi', password='raspberry@',
              db='awesome', maxsize=10,minsize=1,loop=loop)
    with (yield from  pool) as conn:
        cursor=yield from conn.cursor()
        user = User(name=f.first_name(),email=f.email(),passwd=f.state(),image=f.company())
        args = list(map(user.getValueOrDefault, user.__fields__))
        args.append(user.getValueOrDefault(user.__primary_key__))
        yield from cursor.execute(user.__insert__.replace('?','%s') , args )
        yield from cursor.close()
        yield from conn.commit()
    pool.close()
    yield from pool.wait_closed()


loop.run_until_complete(test2())
loop.close()

一切正常,没有翻车。

INFO:root:found model: User (table:users)
INFO:root: found mapping: name --> <StringField,varchar(50):None>
INFO:root: found mapping: email --> <StringField,varchar(50):None>
INFO:root: found mapping: image --> <StringField,varcahr(50):None>
INFO:root: found mapping: id --> <StringField,varcahr(50:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: passwd --> <StringField,varcahr(50):None>
INFO:root: found mapping: admin --> <BooleanField,boolean:None>
INFO:root:found model: Blog (table:blogs)
INFO:root: found mapping: name --> <StringField,varchar(50:None>
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: user_id --> <StringField,varchar(50:None>
INFO:root: found mapping: user_name --> <StringField,varchar(50:None>
INFO:root: found mapping: id --> <StringField,varchar(50):None>
INFO:root: found mapping: user_image --> <StringField,varchar(500:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: summary --> <StringField,varchar(200:None>
INFO:root:found model: Comment (table:comments)
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: user_image --> <StringField,varchar(500):None>
INFO:root: found mapping: id --> <StringField,varchar(50:None>
INFO:root: found mapping: user_name --> <StringField,varchar(50):None>
INFO:root: found mapping: blog_id --> <StringField,varchar(50):None>
INFO:root: found mapping: user_id --> <StringField,varchar(50):None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root:name
INFO:root:email
INFO:root:image
INFO:root:created_at
INFO:root:passwd
INFO:root:admin
INFO:root:id

Process finished with exit code 0

所以,请问这是为什么?

按其他网友说的在loop.close()后加入一个sys.exit()语句,并没有什么卵用。

loop.close()
if loop.is_closed():
    sys.exit(0)

该出错的还出错,我怀疑你们说的治好了都是错觉。

Exception ignored in: <bound method Connection.__del__ of <aiomysql.connection.Connection object at 0x02BE2F10>>
Traceback (most recent call last):
  File "D:\Python35-32\lib\site-packages\aiomysql\connection.py", line 694, in __del__
  File "D:\Python35-32\lib\site-packages\aiomysql\connection.py", line 260, in close
  File "D:\Python35-32\lib\asyncio\selector_events.py", line 573, in close
  File "D:\Python35-32\lib\asyncio\base_events.py", line 497, in call_soon
  File "D:\Python35-32\lib\asyncio\base_events.py", line 506, in _call_soon
  File "D:\Python35-32\lib\asyncio\base_events.py", line 334, in _check_closed
RuntimeError: Event loop is closed

今天上午发现了这个提问,不知是哪位学长提问的,给你点个赞。在这个问题下恰好有朋友给出了答案。 >>去stackoverflow查看原始问题

Before closing event loop, you need to close connection pool, see docs 在关闭event loop之前,首先需要关闭连接池。

pool.close()
yield from pool.wait_closed()
in your case:

loop = asyncio.get_event_loop()
loop.run_until_complete(test_update(loop))
__pool.close()
loop.run_until_complete(__pool.wait_closed())
loop.close()

这个解决方案和我昨天做的试验2思路一致。 所以我决定给orm.py 增加一款销毁连接池的方法。

orm.py

@asyncio.coroutine
def destory_pool(): #销毁连接池
    global __pool
    if __pool is not None:
        __pool.close()
        yield from  __pool.wait_closed()
from .orm import create_pool,destory_pool
#测试方法1
@asyncio.coroutine
def test1():
    yield from create_pool(loop,host='192.168.99.2',port=3306,user='pi',password='raspberry@',db='awesome')
    user = User(name=f.first_name(),email=f.email(),passwd=f.state(),image=f.company())
    yield from user.save()
    yield from destory_pool() #这里先销毁连接池



loop.run_until_complete(test1())
loop.close() #然后从容地关闭event loop

很好,和想象中一样,没有翻车。

INFO:root:found model: User (table:users)
INFO:root: found mapping: admin --> <BooleanField,boolean:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: id --> <StringField,varcahr(50:None>
INFO:root: found mapping: passwd --> <StringField,varcahr(50):None>
INFO:root: found mapping: name --> <StringField,varchar(50):None>
INFO:root: found mapping: email --> <StringField,varchar(50):None>
INFO:root: found mapping: image --> <StringField,varcahr(50):None>
INFO:root:found model: Blog (table:blogs)
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: user_id --> <StringField,varchar(50:None>
INFO:root: found mapping: user_name --> <StringField,varchar(50:None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: id --> <StringField,varchar(50):None>
INFO:root: found mapping: name --> <StringField,varchar(50:None>
INFO:root: found mapping: user_image --> <StringField,varchar(500:None>
INFO:root: found mapping: summary --> <StringField,varchar(200:None>
INFO:root:found model: Comment (table:comments)
INFO:root: found mapping: user_id --> <StringField,varchar(50):None>
INFO:root: found mapping: user_name --> <StringField,varchar(50):None>
INFO:root: found mapping: created_at --> <FloatField,real:None>
INFO:root: found mapping: id --> <StringField,varchar(50:None>
INFO:root: found mapping: content --> <TextField,text:None>
INFO:root: found mapping: user_image --> <StringField,varchar(500):None>
INFO:root: found mapping: blog_id --> <StringField,varchar(50):None>
INFO:root: create databases connection pool...
INFO:root:admin
INFO:root:created_at
INFO:root:passwd
INFO:root:name
INFO:root:email
INFO:root:image
INFO:root:id
INFO:root:SQL: insert into `users` (`admin`, `created_at`, `passwd`, `name`, `email`, `image`,`id`) values (?, ?, ?, ?, ?, ?, ?)
INFO:root:...affected: 1

Process finished with exit code 0

至此破案了,关闭event loop之前首先关闭连接池,问题终结。

多谢你的破案,very helpful

简单粗暴的关闭连接池合适么。。 这个问题最主要的原因就是:执行完SQL语句,没有释放与数据库的连接。

只需要在 orm.py的execute方法中,加上对连接的释放即可:

@asyncio.coroutine
def execute(sql, args):
    log(sql)
    with (yield from __pool) as conn:
        try:
            cur = yield from conn.cursor()
            yield from cur.execute(sql.replace('?', '%s'), args)
            affected = cur.rowcount
            yield from cur.close()
        except BaseException as e:
            raise
        finally:
            conn.close()
        return affected

感谢 @用户6016067289

我刚才复现了那个event loop is closed错误,用你的办法果然很管用,比国际友人的解决方法要好。

我那时候吃亏例程里的代码和github不太同步,我没去看廖大github上的代码。 新来的朋友们请直接去看廖大的github。

@用户6016067289 能不能麻烦讲一讲这到底是什么原理?

我看aiomysql Pool部分的文档说,with pool as conn:这种用法相当于先acquire() ,再release()

类似
with (yield from pool) as conn:
    cur = yield from conn.cursor()

相当于 
 conn = Pool.acquire()
 ...conn 各种操作....
  Pool.release(conn)

你的解决方案里用到了conn.close(), 我估计从pool取出来的connection是被重新包装过的. 它的close()应该是把conn重新归还到pool里,而不是真正去关闭它。 同样是把conn归还到pool里,它比pool.release(conn)还多做了哪些事情?

假如conn.close()就是直接立即关闭连接的话,那pool每次都要重新创建connection,岂不是成了摆设? 想不通,求指点。

可是按照用户6016067289的方法,执行速度,慢了好多好多啊,不如直接关闭连接池快,这两种方式到底有什么区别?原理是什么?

韦德海vvv

#9 Created at ... [Delete] [Delete and Lock User]

@匿名sina网友V,能加个QQ吗,前面有些地方不明白

@匿名sina网友V 你再去看看release(),它的作用是释放pool,让它可以再接收别的连接,而不是关闭pool 即relesase不等于close 所以当然要进行conn.close()或pool.close()了


  • 1
  • 2

Reply