{"componentChunkName":"component---src-templates-blog-detail-tsx","path":"/blog/2020-04-23-serverless-kaggle-scf","result":{"data":{"currentBlog":{"id":"5c34df1d-ef7f-5467-afe7-d0a28d5e8990","frontmatter":{"thumbnail":"https://img.serverlesscloud.cn/2020523/1590213128368-16200.jpg","authors":["乂乂又又"],"categories":["user-stories"],"date":"2020-04-23T00:00:00.000Z","title":"万物皆可 Serverless 之云函数 SCF+Kaggle 端到端验证码识别从训练到部署","description":"今天本文就尝试带大家借助 Kaggle+SCF，快速训练部署一个端到端的通用验证码识别模型，真正的验证码识别从入门到应用的一条龙服务。","authorslink":["https://cloud.tencent.com/developer/article/1618583"],"translators":null,"translatorslink":null,"tags":["云函数","Kaggle"],"keywords":"Serverless,Serverless AI,Serverless应用","outdated":null},"wordCount":{"words":322,"sentences":52,"paragraphs":49},"fileAbsolutePath":"/opt/build/repo/content/blog/2020-04-23-serverless-kaggle-scf.md","fields":{"slug":"/blog/2020-04-23-serverless-kaggle-scf/","keywords":["go","java","python","serverless","云函数","验证码","kaggle","模型","识别","训练","serverlesscloud"]},"html":"

随着验证码技术的更新换代，传统的验证码识别算法已经越来越无用武之地了。近些年来人工智能迅速发展，尤其是在深度学习神经网络这一块生态尤为繁荣，各种算法和模型层出不穷。

\n

今天本文就尝试带大家借助 Kaggle+SCF 快速训练部署一个端到端的通用验证码识别模型，真正的验证码识别从入门到应用的一条龙服务，哈哈哈~

\n

效果展示

\n

\"正在

\n

\"调用训练好的模型识别验证码\"

\n

操作步骤

\n

第一步：了解 kaggle

\n

没做过数据科学竞赛的同学，可能不太了解 kaggle 哈。

\n
\n

Kaggle is the world’s largest data science community with powerful tools and resources to help you achieve your data science goals.

\n
\n

这是 kaggle 官网）的自我介绍，简单来说 kaggle 是全球最大的数据科学交流社区，上面有许多关于数据科学的竞赛和数据集，并且提供了一些数据科学在线分析的环境和工具，一直以来吸引了全球大批数据科学爱好者，社区极其繁荣。

\n

这里我们主要是用 kaggle 的 Notebooks 服务里的 kernel 环境来快速在云端训练自己的验证码识别模型。

\n

\"一个kernel实例\"

\n

你可能会问在本地训练不可以吗，为啥非得折腾着上云？哈哈，这还真不是折腾，普通人的电脑算力其实是有限的，而训练模型是需要强大 GPU 算力的支持，不然要训练到猴年马月~

\n

\"训练模型时

\n

我们再来看一下 kaggle 上的 kernel 环境的配置：

\n\n

每个 kernel 有 9 小时的运行时长，GPU 资源每周 30 小时使用时长。除了硬件资源之外，kernel 环境里已经配置好了一些机器学习的常用库，包括 Pytorch， Tensorflow 2 等，它的机器学习环境是开箱即用的，零配置，零维护。

\n
\n

Kaggle Notebooks run in a remote computational environment. We provide the hardware—you need only worry about the code.

\n
\n

正如 kaggle notebooks 官方文档所言，kaggle 免费为你提供硬件和机器学习环境，你唯一需要关心的是你的代码。这么好的东西关键还是免费提供的啊，果断选它来训练模型就对了。

\n

第二步：注册 kaggle 账号，新建一个 kernel 环境

\n

账号注册、新建 kernel 等相关问题，网上有很多相关文章，这里不再细说了。

\n

第三步：clone git 仓库，修改成自己的验证码数据集

\n

\"模型训练仓库\"

\n

这里我在 github.com/nickliqian/cnn_captcha项目的基础上，把原项目升级更新到了 Tensorflow 2.0，然后做了个 kaggle 训练 + SCF 部署的通用验证码识别方案。

\n

现在你只需要将我修改好的仓库 https://gitee.com/LoveWJG/tflite_train克隆到本地，

\n

然后按照项目里的 readme 配置一下训练参数，替换一下自己的验证码数据集即可。

\n

第四步：上传项目到 kaggle 开始训练

\n

然后把配置好的项目压缩上传到 kaggle 直接解压按照说明文件进行训练即可。

\n

\"模型训练中\"

\n

这里用了 20000 张验证码，训练了 10000 轮左右，大概耗时 30 分钟，还是相当给力的。训练结束后你可以根据仓库里的 readme 文件，把模型、日志文件打包下载到本地，然后再在本地将模型转成 tflite 格式（方便在移动端使用，本地识别验证码），如果模型文件过大你也可以在本地运行 tflite.py 程序把 tflite 模型量化，大概可以把模型文件缩小到原来的 1/4，最终你应该得到一个 .tflite 格式的模型文件。

\n

\"最终的模型文件\"

\n

第五步：使用云函数快速部署验证码识别模型

\n

云函数的创建、配置和发布可参考我之前的系列文章，这里就不再细讲了。

\n

新建一个 python 空白云函数，然后把 scf.py 文件里的代码填到 index.py 里保存。

\n\n \n 复制代码\n \n \n
# -*- coding:utf-8 -*-\nimport io\nimport json\nimport os\nimport time\n\nimport numpy as np\n\nimport tensorflow as tf\nfrom PIL import Image\n\nmodel_path = \"model_quantized.tflite\" #模型文件地址\nchars = '23456789abcdefghjkmpqrstuvwxy' #验证码字符，顺序要与config.json里的一致\n\n# Load TFLite model and allocate tensors.\ninterpreter = tf.lite.Interpreter(model_path=model_path)\ninterpreter.allocate_tensors()\n\n# Get input and output tensors.\ninput_details = interpreter.get_input_details()\noutput_details = interpreter.get_output_details()\n\n#将验证码数据转换成模型输入格式\ndef img2input(img, width, height):\n    tmpe_array = [serverless]\n    for i in range(height):\n        for j in range(width):\n            pixel = img.getpixel((j, i))\n            tmpe_array.append((0.3*pixel[0]+0.6*pixel[1]+0.1*pixel[2])/255)\n    tmpe_array = np.array(tmpe_array).astype('float32')\n    input_array = np.expand_dims(tmpe_array, axis=0)\n    return input_array\n\n#识别验证码\ndef predict(image):\n    captcha_image = Image.open(io.BytesIO(image))\n    image_np_expanded = img2input(captcha_image, 100, 50)\n    interpreter.set_tensor(input_details[0]['index'], image_np_expanded)\n    interpreter.invoke()\n    output_data = interpreter.get_tensor(output_details[0]['index'])\n    codes = ''\n    for i in output_data[0]:\n        codes += chars[i]\n    return codes\n\n\n# api网关响应集成\ndef apiReply(reply, txt=False, content_type='application/json', code=200):\n    return {\n        \"isBase64Encoded\": False,\n        \"statusCode\": code,\n        \"headers\": {'Content-Type': content_type},\n        \"body\": json.dumps(reply, ensure_ascii=False) if not txt else str(reply)\n    }\n\n#云函数入口\ndef main_handler(event, context):\n    return apiReply(\n        {\n            \"ok\": False if not 'image' in event.keys() else True,\n            \"message\": \"请求参数无效\" if not 'image' in event.keys() else predict(event['queryString']['image'])\n        }\n    )
\n

把模型文件上传到云函数根目录，然后配置一下自己的验证码识别模型参数

\n\n \n 复制代码\n \n \n
model_path = \"model_quantized.tflite\" #模型文件地址\nchars = '23456789abcdefghjkmpqrstuvwxy' #验证码字符，顺序要与config.json里的一致
\n

之后给我们的云函数添加一个 API 网关触发器，并启用响应集成，然后发布上线即可

\n

\"最终结果\"

\n

没有问题的话，你只需 GET 一下，就可以返回验证码识别结果了。

\n\n \n 复制代码\n \n \n
api网关+?base64Image=base64编码后的验证码数据
\n

写在最后

\n

本文带大家从头训练并部署了一个通用验证码识别模型。我们再一次看到基于 Serverless 的云函数在开发线上应用的过程中是多么方便和迅速！

\n

如果你对验证码识别比较感兴趣，想要了解更多的识别方案，这里我推荐几个 github 仓库，都是可以直接上手应用的程度。

\n
    \n
  • use cnn recognize captcha by tensorflow 本项目针对字符型图片验证码，使用 tensorflow 实现卷积神经网络，进行验证码识别。
    • \n
  • 验证码识别-训练 This project is based on CNN/ResNet/DenseNet+GRU/LSTM+CTC/CrossEntropy to realize verification code identification. This project is only for training the model.
    • \n
  • 验证码识别-部署 This project is based on CNN+BLSTM+CTC to realize verificationtion. This projeccode identificat is only for deployment models.
    • \n
\n
\n
\n
\n
\n

传送门：

\n\n
\n

欢迎访问：Serverless 中文网，您可以在 最佳实践 里体验更多关于 Serverless 应用的开发！

","tableOfContents":""},"previousBlog":{"id":"5534f888-0f20-5eed-9d9f-265a12f0958e","frontmatter":{"thumbnail":"https://serverlessimg-1253970226.cos.ap-chengdu.myqcloud.com/qianyi/images/162020.jpg","authors":["乂乂又又"],"categories":["user-stories"],"date":"2020-04-23T00:00:00.000Z","title":"万物皆可 Serverless 之免费搭建不限速 5T 大云盘","description":"不晓得你有没有体验过百度云限速的痛苦，反正我对网盘限速这件事一直深恶痛绝，我行我上，走起！","authorslink":["https://cloud.tencent.com/developer/article/1612098"],"translators":null,"translatorslink":null,"tags":["Serverless","网盘"],"keywords":"Serverless,Serverless实践,Serverless应用","outdated":null},"wordCount":{"words":283,"sentences":83,"paragraphs":83},"fileAbsolutePath":"/opt/build/repo/content/blog/2020-04-23-serverless-cloud-cos.md","fields":{"slug":"/blog/2020-04-23-serverless-cloud-cos/","keywords":["go","php","serverless","云函数","serverlesscloud","OneDrive","函数","网关"]}},"nextBlog":{"id":"07918513-5a0f-5c41-b33c-209be7389654","frontmatter":{"thumbnail":"https://img.serverlesscloud.cn/202068/1591592273724-70zt6hhv58%20%281%29.png","authors":["乂乂又又"],"categories":["user-stories"],"date":"2020-04-23T00:00:00.000Z","title":"万物皆可 Serverless 之使用云函数 SCF+COS 快速开发全栈应用","description":"我是在一个前端初学者的背景下，前后仅花了大概三天的时间，就完成了一个比较简单的网页应用，这就是 Severless 的魅力所在，它可以让你快速开发上线全栈应用，无论你是前端或是后端开发者都可以获益许多。","authorslink":["https://cloud.tencent.com/developer/article/1612750"],"translators":null,"translatorslink":null,"tags":["全栈应用","云函数"],"keywords":"Serverless,Serverless全栈,Serverless应用","outdated":true},"wordCount":{"words":340,"sentences":82,"paragraphs":82},"fileAbsolutePath":"/opt/build/repo/content/blog/2020-04-23-serverless-scf-cos.md","fields":{"slug":"/blog/2020-04-23-serverless-scf-cos/","keywords":["java","python","serverless","spa","腾讯云无服务器云函数","无服务器","无服务器云函数","云函数","函数","cos","json","serverlesscloud","event","blackMap"]}},"recommendBlogs":{"edges":[{"node":{"id":"4300b21c-7209-5256-86ff-0d38e3daec9b","frontmatter":{"thumbnail":"https://main.qcloudimg.com/raw/14f1c8eed372e76c1b139703b2f6d0fa.jpg","authors":["KieranMcCarthy"],"categories":["user-stories","engineering-culture"],"date":"2018-01-09T00:00:00.000Z","title":"我是如何在四年时间里，从厨师转行为 Serverless 应用开发者","description":"我是厨师出身，现在成为了一名 Serverless 应用开发者。","authorslink":["https://serverless.com/author/kieranmccarthy/"],"translators":["Aceyclee"],"translatorslink":["https://www.zhihu.com/people/Aceyclee"],"tags":["应用开发","Serverless"],"keywords":"Serverless 应用开发,Serverless 管理,厨师转行为 Serverless 应用开发者","outdated":null},"wordCount":{"words":285,"sentences":38,"paragraphs":36},"fileAbsolutePath":"/opt/build/repo/content/blog/2018-01-09-from-chef-to-serverless-developer-in-4-years.md","fields":{"slug":"/blog/2018-01-09-from-chef-to-serverless-developer-in-4-years/","keywords":["无服务器","无服务器开发","云函数","学习","Serverless","构建","Framework","开发者","服务器","应用","学位","简历"]}}},{"node":{"id":"713a0563-4bf9-5721-bacb-3b4ef609fe4a","frontmatter":{"thumbnail":"https://s3-us-west-2.amazonaws.com/assets.blog.serverless.com/camp-fire/camp-fire-housing-thumb.jpg","authors":["EricWyne"],"categories":["guides-and-tutorials","user-stories"],"date":"2018-12-05T00:00:00.000Z","title":"Serverless Twitter 机器人帮助为坎普山火受灾者安置住房","description":"加利福尼亚州的坎普山火致使数千人流离失所，为此，我构建了一个简单的 Serverless Twitter 机器人来帮助将受灾者安置在临时住房！","authorslink":["https://serverless.com/author/ericwyne/"],"translators":["Aceyclee"],"translatorslink":["zhihu.com/people/Aceyclee"],"tags":null,"keywords":null,"outdated":null},"wordCount":{"words":157,"sentences":26,"paragraphs":26},"fileAbsolutePath":"/opt/build/repo/content/blog/2018-12-05-serverless-twitter-camp-fire.md","fields":{"slug":"/blog/2018-12-05-serverless-twitter-camp-fire/","keywords":["serverless","无服务器","云函数","Serverless","org","住房","Twitter","函数","受灾","机器人","山火"]}}},{"node":{"id":"98602143-b837-5f50-a24f-3b1ec76044d7","frontmatter":{"thumbnail":"https://s3-us-west-2.amazonaws.com/assets.blog.serverless.com/sqquid/sqquid-serverless-thumb.jpg","authors":["RonPeled"],"categories":["user-stories"],"date":"2018-12-17T00:00:00.000Z","title":"SQQUID：100% 无服务器初创公司","description":"SQQUID 将 AWS Lambda 和无服务器框架用于其核心产品和营销网站。我们来看看一个完全无服务器的初创公司是怎样的。","authorslink":null,"translators":null,"translatorslink":null,"tags":null,"keywords":null,"outdated":null},"wordCount":{"words":266,"sentences":42,"paragraphs":42},"fileAbsolutePath":"/opt/build/repo/content/blog/2018-12-17-sqquid-one-hundred-percent-serverless.md","fields":{"slug":"/blog/2018-12-17-sqquid-one-hundred-percent-serverless/","keywords":["go","serverless","无服务器","无服务器架构","服务器","架构","Lambda","集成","FaaS","串行","系统"]}}},{"node":{"id":"29dc2e58-d2ba-56f9-aee1-d21b0bc62e0e","frontmatter":{"thumbnail":"https://s3-us-west-2.amazonaws.com/assets.blog.serverless.com/ao-com-story/ao-serverless-thumbnail.png","authors":["NickGottlieb"],"categories":["user-stories"],"date":"2019-04-24T00:00:00.000Z","title":"AO.com：逐渐转向无服务器优先","description":"AO.com 的 SCV 团队率先尝试无服务器服务。折服于无服务器框架的快速周转时间和低维护成本，整个团队逐渐转向无服务器优先。","authorslink":null,"translators":null,"translatorslink":null,"tags":null,"keywords":null,"outdated":null},"wordCount":{"words":236,"sentences":42,"paragraphs":35},"fileAbsolutePath":"/opt/build/repo/content/blog/2019-04-24-ao-serverless-first.md","fields":{"slug":"/blog/2019-04-24-ao-serverless-first/","keywords":["serverless","无服务器","服务器","团队","Lambda","功能","构建"]}}},{"node":{"id":"752d08d1-387a-5bde-acf3-98141baab294","frontmatter":{"thumbnail":"https://img.serverlesscloud.cn/2020414/1586871710979-%E5%85%AC%E5%85%B1%E7%94%A8.png","authors":["Anycodes"],"categories":["user-stories"],"date":"2019-06-20T00:00:00.000Z","title":"如何用 Serverless 为 Python 云函数打包依赖","description":"在使用无服务器云函数SCF时通常会遇到导入第三方库的问题，很多小伙伴比较头疼是：应该如何打包进去？这里，推荐几个不错的方法。","authorslink":["https://zhuanlan.zhihu.com/ServerlessGo"],"translators":null,"translatorslink":null,"tags":["云函数","Serverless"],"keywords":"Serverless,Serverless应用,无服务器云函数","outdated":null},"wordCount":{"words":81,"sentences":43,"paragraphs":43},"fileAbsolutePath":"/opt/build/repo/content/blog/2019-06-20-for-python-cloud-functions.md","fields":{"slug":"/blog/2019-06-20-for-python-cloud-functions/","keywords":["java","serverless","无服务器","无服务器云函数","云函数","serverlesscloud","安装","serverless","pillowtest"]}}},{"node":{"id":"2dc78814-9d77-555b-a1bb-ad202c8ec2d1","frontmatter":{"thumbnail":"https://s3-us-west-2.amazonaws.com/assets.blog.serverless.com/cloudforecast/thumbnail.png","authors":["FrancoisLagier"],"categories":["user-stories"],"date":"2019-08-07T00:00:00.000Z","title":"Serverless：初创企业的理想选择？（CloudForecast 案例分析）","description":"CloudForecast 是 2018 年成立的一家独立初创企业，本文将介绍他们决定选择 Serverless 的原因。","authorslink":["https://serverless.com/author/francoislagier/"],"translators":["Aceyclee"],"translatorslink":["zhihu.com/people/Aceyclee"],"tags":null,"keywords":null,"outdated":null},"wordCount":{"words":211,"sentences":29,"paragraphs":29},"fileAbsolutePath":"/opt/build/repo/content/blog/2019-08-07-serverless-for-startups.md","fields":{"slug":"/blog/2019-08-07-serverless-for-startups/","keywords":["serverless","云函数","serverless","函数","Serverless","utm","Framework","blog","CloudForecast","cloudforecast"]}}},{"node":{"id":"97450b07-658b-5207-8216-1c7b9b51b115","frontmatter":{"thumbnail":"https://img.serverlesscloud.cn/2020114/1578988490344-v2-8b2cd2c5275aa2c5a3c5083a148a7a9f_1200x500.jpg","authors":["Anycodes"],"categories":["user-stories"],"date":"2019-09-01T00:00:00.000Z","title":"如何通过 Serverless 与自然语言处理，让搜索引擎「看」到你的博客","description":"Serverless 与自然语言处理结合的一个小应用","authorslink":["https://www.zhihu.com/people/liuyu-43-97"],"translators":null,"translatorslink":null,"tags":["个人博客","serverless"],"keywords":"Serverless 自然语言处理","outdated":null},"wordCount":{"words":106,"sentences":34,"paragraphs":34},"fileAbsolutePath":"/opt/build/repo/content/blog/2019-09-01-search-engine-blog.md","fields":{"slug":"/blog/2019-09-01-search-engine-blog/","keywords":["serverless","云函数","keywords","serverlesscloud","summary"]}}},{"node":{"id":"ae4fd2f8-515c-5aec-b584-38427ef33f7e","frontmatter":{"thumbnail":"https://img.serverlesscloud.cn/2020114/1578989800047-part-00492-780.jpg","authors":["Anycodes"],"categories":["guides-and-tutorials","user-stories"],"date":"2019-09-16T00:00:00.000Z","title":"突破传统 OJ 瓶颈，「判题姬」接入云函数","description":"通过 Serverless 实现在线编程","authorslink":["https://www.zhihu.com/people/liuyu-43-97"],"translators":null,"translatorslink":null,"tags":["在线编程","云函数"],"keywords":"Serverless 在线编程,Serverless OJ","outdated":null},"wordCount":{"words":169,"sentences":30,"paragraphs":30},"fileAbsolutePath":"/opt/build/repo/content/blog/2019-09-16-online-Judge.md","fields":{"slug":"/blog/2019-09-16-online-Judge/","keywords":["python","serverless","云函数","代码","函数","serverless"]}}}],"totalCount":64}},"pageContext":{"isCreatedByStatefulCreatePages":false,"blogId":"5c34df1d-ef7f-5467-afe7-d0a28d5e8990","previousBlogId":"5534f888-0f20-5eed-9d9f-265a12f0958e","nextBlogId":"07918513-5a0f-5c41-b33c-209be7389654","categories":["user-stories"]}}}