当前位置:  互联网>综合

[1]决策树绘图（二）

由于最近在看机器学习实战，所以自己利用python3去完成里面的代码，此代码衔接着http://blog.csdn.net/xueyunf/article/details/9223865。

在这个基础上进行修改完成了这篇文章的代码，我们知道了决策树的简单构建，ID3算法完成，当然这都很基础，画图呢，只是为了让其更加形象化；我们添加几个函数，一个是输出一棵我们可以利用ID3算法生成的树，一个获取树的叶子节点，一个获取树的深度，这些我想这里就不用讲解了，学过数据结构的童鞋，可以在非常短的时间内实现这些算法；当然我先把这3个函数的代码贴出来：

def  getNumLeafs(myTree):
    numLeafs = 0
    firstStr = list(myTree.keys())[0]
    secondDict  =  myTree[firstStr]
    for key in secondDict.keys():
        if type(secondDict[key]).__name__=='dict':
            numLeafs += getNumLeafs(secondDict[key])
        else:
            numLeafs += 1
    return numLeafs
    
def getTreeDepth(myTree):
    maxDepth = 0
    firstStr = list(myTree.keys())[0]
    secondDict = myTree[firstStr]
    for key in secondDict.keys():
        if type(secondDict[key]).__name__=='dict':
            thisDepth = 1 + getTreeDepth(secondDict[key])
        else:
            thisDepth = 1
        if thisDepth>maxDepth:
            maxDepth = thisDepth
    return maxDepth

def retrieveTree(i):
    listOfTrees = [{'no surfacing':{0:'no', 1:{'flippers':\
                    {0:'no', 1:'yes'}}}},
                   {'no surfacing':{0:'no', 1:{'flippers':\
                      {0:{'head':{0:'no', 1:'yes'}}, 1:'no'}}}}
                ] 
                
    return listOfTrees[i]

我们不难看出根据定义第一个函数完成获取所有叶子节点的个数，第二个函数完成获取树的高度，第三个函数完成输出树。

然后我们放出这次的主要函数，修改后的绘图函数：

def plotMidText(cntrPt, parentPt, txtString):
    xMid = (parentPt[0]-cntrPt[0])/2.0 + cntrPt[0]
    yMid = (parentPt[1]-cntrPt[1])/2.0 + cntrPt[1]
    createPlot.ax1.text(xMid, yMid, txtString)

def plotTree(myTree, parentPt, nodeTxt):
    numLeafs = getNumLeafs(myTree)
    getTreeDepth(myTree)
    firstStr = list(myTree.keys())[0]
    cntrPt = (plotTree.xOff + (1.0 + float(numLeafs))/2.0/plotTree.totalW,\
    plotTree.yOff)
    plotMidText(cntrPt, parentPt, nodeTxt)
    plotNode(firstStr, cntrPt, parentPt, decisionNode)
    secondDict = myTree[firstStr]
    plotTree.yOff = plotTree.yOff - 1.0/plotTree.totalD
    for key in secondDict.keys():
        if type(secondDict[key]).__name__=='dict':
            plotTree(secondDict[key],cntrPt,str(key))
        else:
            plotTree.xOff = plotTree.xOff + 1.0/plotTree.totalW
            plotNode(secondDict[key], (plotTree.xOff, plotTree.yOff),
            cntrPt, leafNode)
            plotMidText((plotTree.xOff, plotTree.yOff), cntrPt, str(key))
    plotTree.yOff = plotTree.yOff + 1.0/plotTree.totalD

def createPlot(inTree):
    fig = plt.figure(1, facecolor='white')
    fig.clf()
    axprops = dict(xticks=[], yticks=[])
    createPlot.ax1 = plt.subplot(111, frameon=False, **axprops)
    plotTree.totalW = float(getNumLeafs(inTree))
    plotTree.totalD = float(getTreeDepth(inTree))
    plotTree.xOff = -0.5/plotTree.totalW; plotTree.yOff = 1.0;
    plotTree(inTree, (0.5,1.0), '')
    plt.show()

最后当然也是截个图给大家看看程序的运行情况：

好了，这里面的函数我想大家可以通过名字也知道每个函数干了些什么。

[2]zookeeper集群的python代码测试

上一篇已经讲解了如何安装zookeeper的python客户端，接下来是我在网上搜到的例子，举例应用环境是：

1.当有两个或者多个服务运行，并且同意时间只有一个服务接受请求(工作)，其他服务待命。

2.当接受请求(工作)的服务异常挂掉时，会从剩下的待命服务中选举出一个服务来接受请求(工作)。

下面直接上例子，有两个文件组成1.zkclient.py   2.zktest.py

# coding: utf-8
# modfied from https://github.com/phunt/zk-smoketest/blob/master/zkclient.py
# zkclient.py

import zookeeper, time, threading
from collections import namedtuple

DEFAULT_TIMEOUT = 30000
VERBOSE = True

ZOO_OPEN_ACL_UNSAFE = {"perms":0x1f, "scheme":"world", "id" :"anyone"}

# Mapping of connection state values to human strings.
STATE_NAME_MAPPING = {
    zookeeper.ASSOCIATING_STATE: "associating",
    zookeeper.AUTH_FAILED_STATE: "auth-failed",
    zookeeper.CONNECTED_STATE: "connected",
    zookeeper.CONNECTING_STATE: "connecting",
    zookeeper.EXPIRED_SESSION_STATE: "expired",
}

# Mapping of event type to human string.
TYPE_NAME_MAPPING = {
    zookeeper.NOTWATCHING_EVENT: "not-watching",
    zookeeper.SESSION_EVENT: "session",
    zookeeper.CREATED_EVENT: "created",
    zookeeper.DELETED_EVENT: "deleted",
    zookeeper.CHANGED_EVENT: "changed",
    zookeeper.CHILD_EVENT: "child", 
}

class ZKClientError(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)

class ClientEvent(namedtuple("ClientEvent", 'type, connection_state, path')):
    """
    A client event is returned when a watch deferred fires. It denotes
    some event on the zookeeper client that the watch was requested on.
    """

    @property
    def type_name(self):
        return TYPE_NAME_MAPPING[self.type]

    @property
    def state_name(self):
        return STATE_NAME_MAPPING[self.connection_state]

    def __repr__(self):
        return  "<ClientEvent %s at %r state: %s>" % (
            self.type_name, self.path, self.state_name)


def watchmethod(func):
    def decorated(handle, atype, state, path):
        event = ClientEvent(atype, state, path)
        return func(event)
    return decorated

class ZKClient(object):
    def __init__(self, servers, timeout=DEFAULT_TIMEOUT):
        self.timeout = timeout
        self.connected = False
        self.conn_cv = threading.Condition( )
        self.handle = -1

        self.conn_cv.acquire()
        if VERBOSE: print("Connecting to %s" % (servers))
        start = time.time()
        self.handle = zookeeper.init(servers, self.connection_watcher, timeout)
        self.conn_cv.wait(timeout/1000)
        self.conn_cv.release()

        if not self.connected:
            raise ZKClientError("Unable to connect to %s" % (servers))

        if VERBOSE:
            print("Connected in %d ms, handle is %d"
                  % (int((time.time() - start) * 1000), self.handle))

    def connection_watcher(self, h, type, state, path):
        self.handle = h
        self.conn_cv.acquire()
        self.connected = True
        self.conn_cv.notifyAll()
        self.conn_cv.release()

    def close(self):
        return zookeeper.close(self.handle)
    
    def create(self, path, data="", flags=0, acl=[ZOO_OPEN_ACL_UNSAFE]):
        start = time.time()
        result = zookeeper.create(self.handle, path, data, acl, flags)
        if VERBOSE:
            print("Node %s created in %d ms"
                  % (path, int((time.time() - start) * 1000)))
        return result

    def delete(self, path, version=-1):
        start = time.time()
        result = zookeeper.delete(self.handle, path, version)
        if VERBOSE:
            print("Node %s deleted in %d ms"
                  % (path, int((time.time() - start) * 1000)))
        return result

    def get(self, path, watcher=None):
        return zookeeper.get(self.handle, path, watcher)

    def exists(self, path, watcher=None):
        return zookeeper.exists(self.handle, path, watcher)

    def set(self, path, data="", version=-1):
        return zookeeper.set(self.handle, path, data, version)

    def set2(self, path, data="", version=-1):
        return zookeeper.set2(self.handle, path, data, version)


    def get_children(self, path, watcher=None):
        return zookeeper.get_children(self.handle, path, watcher)

    def async(self, path = "/"):
        return zookeeper.async(self.handle, path)

    def acreate(self, path, callback, data="", flags=0, acl=[ZOO_OPEN_ACL_UNSAFE]):
        result = zookeeper.acreate(self.handle, path, data, acl, flags, callback)
        return result

    def adelete(self, path, callback, version=-1):
        return zookeeper.adelete(self.handle, path, version, callback)

    def aget(self, path, callback, watcher=None):
        return zookeeper.aget(self.handle, path, watcher, callback)

    def aexists(self, path, callback, watcher=None):
        return zookeeper.aexists(self.handle, path, watcher, callback)

    def aset(self, path, callback, data="", version=-1):
        return zookeeper.aset(self.handle, path, data, version, callback)

watch_count = 0

"""Callable watcher that counts the number of notifications"""
class CountingWatcher(object):
    def __init__(self):
        self.count = 0
        global watch_count
        self.id = watch_count
        watch_count += 1

    def waitForExpected(self, count, maxwait):
        """Wait up to maxwait for the specified count,
        return the count whether or not maxwait reached.

        Arguments:
        - `count`: expected count
        - `maxwait`: max milliseconds to wait
        """
        waited = 0
        while (waited < maxwait):
            if self.count >= count:
                return self.count
            time.sleep(1.0);
            waited += 1000
        return self.count

    def __call__(self, handle, typ, state, path):
        self.count += 1
        if VERBOSE:
            print("handle %d got watch for %s in watcher %d, count %d" %
                  (handle, path, self.id, self.count))

"""Callable watcher that counts the number of notifications
and verifies that the paths are sequential"""
class SequentialCountingWatcher(CountingWatcher):
    def __init__(self, child_path):
        CountingWatcher.__init__(self)
        self.child_path = child_path

    def __call__(self, handle, typ, state, path):
        if not self.child_path(self.count) == path:
            raise ZKClientError("handle %d invalid path order %s" % (handle, path))
        CountingWatcher.__call__(self, handle, typ, state, path)

class Callback(object):
    def __init__(self):
        self.cv = threading.Condition()
        self.callback_flag = False
        self.rc = -1

    def callback(self, handle, rc, handler):
        self.cv.acquire()
        self.callback_flag = True
        self.handle = handle
        self.rc = rc
        handler()
        self.cv.notify()
        self.cv.release()

    def waitForSuccess(self):
        while not self.callback_flag:
            self.cv.wait()
        self.cv.release()

        if not self.callback_flag == True:
            raise ZKClientError("asynchronous operation timed out on handle %d" %
                             (self.handle))
        if not self.rc == zookeeper.OK:
            raise ZKClientError(
                "asynchronous operation failed on handle %d with rc %d" %
                (self.handle, self.rc))


class GetCallback(Callback):
    def __init__(self):
        Callback.__init__(self)

    def __call__(self, handle, rc, value, stat):
        def handler():
            self.value = value
            self.stat = stat
        self.callback(handle, rc, handler)

class SetCallback(Callback):
    def __init__(self):
        Callback.__init__(self)

    def __call__(self, handle, rc, stat):
        def handler():
            self.stat = stat
        self.callback(handle, rc, handler)

class ExistsCallback(SetCallback):
    pass

class CreateCallback(Callback):
    def __init__(self):
        Callback.__init__(self)

    def __call__(self, handle, rc, path):
        def handler():
            self.path = path
        self.callback(handle, rc, handler)

class DeleteCallback(Callback):
    def __init__(self):
        Callback.__init__(self)

    def __call__(self, handle, rc):
        def handler():
            pass
        self.callback(handle, rc, handler)

上面的文件是别人封装好的zookeeper接口，下面是测试代码，需要依赖上面的包：

# coding: utf-8
# zktest.py

import logging
from os.path import basename, join

from zkclient import ZKClient, zookeeper, watchmethod

logging.basicConfig(
    level = logging.DEBUG,
    format = "[%(asctime)s] %(levelname)-8s %(message)s"
)

log = logging

class GJZookeeper(object):

    ZK_HOST = "localhost:2181"
    ROOT = "/app"
    WORKERS_PATH = join(ROOT, "workers")
    MASTERS_NUM = 1
    TIMEOUT = 10000

    def __init__(self, verbose = True):
        self.VERBOSE = verbose
        self.masters = []
        self.is_master = False
        self.path = None

        self.zk = ZKClient(self.ZK_HOST, timeout = self.TIMEOUT)
        self.say("login ok!")
        # init
        self.__init_zk()
        # register
        self.register()

    def __init_zk(self):
          

      
      
    

      

[3]用curl访问HTTPS站点并登录

开发网站，少不了测试。现在的网站为了加强安全性，都启用了HTTPS协议。所谓HTTPS，也就是HTTP文本在SSL协议中传输。用curl命令行来测试HTTPS站点是个很有用的功能，写点脚本，就可以做功能测试。

假定Ubuntu系统运行着一个HTTPS站点，用CppCMS编写，Nginx配置了SSL证书，通过FastCGI和CppCMS编写的后台进程连接在一起。

第一步，安装：

apt-get install curl

我的Ubuntu是13.04, 因此安装的curl版本很新，下面的命令检查版本号和其他信息：

curl -V
curl 7.29.0 (x86_64-pc-linux-gnu) libcurl/7.29.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.25 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

我们可以看到启用了SSL, 并且openssl版本是1.0.1c。

第二步，访问HTTP站点：

curl http://www.baidu.com
<!DOCTYPE html><!--STATUS OK--><html><head><meta http-equiv="content-type" content="text/html;charset=utf-8"><title>百度一下，你就知道<unction(){var _t=new Date().getTime();document.cookie = "WWW_ST=" + _t +";expires=" + new Date(_t + 10000).toGMTString()})}catch(e){}</script></html><!--b5d54ba904675fbf-->

返回了百度的网页内容。内容太多，裁剪了。

第三步，查看详细信息，用-v参数。

 curl -v http://www.baidu.com
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 61.135.169.125...
* Connected to www.baidu.com (61.135.169.125) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 03 Jul 2013 13:55:45 GMT
< Server: BWS/1.0
< Content-Length: 10437
< Content-Type: text/html;charset=utf-8
< Cache-Control: private
< Set-Cookie: BDSVRTM=24; path=/
< Set-Cookie: H_PS_PSSID=2757_1457_2704_2726_1788_2249_2702; path=/; domain=.baidu.com
< Set-Cookie: BAIDUID=5E81F8E70C5DE6EDB5C24088E3E56359:FG=1; expires=Wed, 03-Jul-43 13:55:45 GMT; path=/; domain=.baidu.com
< Expires: Wed, 03 Jul 2013 13:55:45 GMT
< P3P: CP=" OTI DSP COR IVA OUR IND COM "
< Connection: Keep-Alive
< 
<!DOCTYPE html><!--STATUS OK--><html><head><meta http-equiv="content-type" content="text/html;charset=utf-8"><title>百度一下，你就知道</title><style >html,body{height:100%}html{overflow-y:auto}#wrapper{position:relative;_position:;min-height:100%}#content{padding-bottom:100px;text-align:center}#ftCon{height:100px;position:absolute;bottom:44px;text-align:center;width:100%;margin:0 auto;z-index:0;overflow:hidden}#ftConw{width:720px;margin:0 auto}body{font:12px arial;text-align:;background:#fff}body,p,form,ul,li{margin:0;padding:0;list-style:none}body,form,#fm{position:relative}td{text-align:left}img{border:0}a{color:#00c}a:active{color:#f60}#u{color:#999;padding:4px 10px 5px 0;text-align:right}#u a{margin:0 5px}#u .reg{margin:0}#m{width:720px;margin:0 auto}#nv a,#nv b,.btn,#lk{font-size:14px}#fm{padding-left:110px;text-align:left;z-index:1}input{border:0;padding:0}#nv{height:19px;font-size:16px;margin:0 0 4px;text-alig

这样详细的信息都显示出来了。-v参数很有用，一般调试时都打开。

如果只想查看头部信息，用-i代替-v.

第四步，访问本地HTTPS站点

curl --insecure https://localhost/your_site/login_page

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
      <meta http-equiv="pragma" content="no-cache">
	<meta http-equiv="cache-control" content="no-cache">

---insecure表示忽略校验步骤。

我试过用--cacert选项指定server.crt文件，也就是我的nginx使用的那个文件。但是报错。所以直接忽略算了。

第五步，调用HTTPS的login API登录

curl -v --insecure -d "email=gche@yousite.com&pwd=123456&language=en" https://localhost/your_site/login
* About to connect() to localhost port 443 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES256-SHA
* Server certificate:
* 	 subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* 	 start date: 2013-06-02 07:24:53 GMT
* 	 expire date: 2014-06-02 07:24:53 GMT
* 	 issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* 	 SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /your_site/login HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
> Content-Length: 51
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 51 out of 51 bytes
< HTTP/1.1 200 OK
< Server: nginx/1.5.1
< Date: Wed, 03 Jul 2013 14:02:38 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: CppCMS/1.0.3
< Set-Cookie: cml_session=518b7fc5117e87bce28f2444; Max-Age=36000; Path=/; Version=1
< 
* Connection #0 to host localhost left intact
{"message":"Login succeeded!","status":0,"value":""}

-d "...&..." 的参数是通过POST方法发送参数。服务端最终回复一个JSON格式的字符串，表示登录成功。并且拿到了cml_session的值，也就是cookie.

第六步，用cookie访问HTTP网页。后面的网页只需要HTTP访问，提供正确的cookie即可。

curl -v --cookie "cml_session=518b7fc5117e87bce28f2444" http://localhost/your_site/home
* About to connect() to localhost port 80 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /your_site/home HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
> Cookie: cml_session=518b7fc5117e87bce28f2444
> 
< HTTP/1.1 200 OK
< Server: nginx/1.5.1
< Date: Wed, 03 Jul 2013 14:06:43 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: CppCMS/1.0.3
< 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
      <meta http-equiv="pragma" content="no-cache">
	<meta http-equiv="cache-control" content="no-cache">
	  <meta http-equiv="expires" content="0">
	    <title>CML Cloud</title>
	    <link type="text/css" href=/blog_article/"/style/reset.css" rel="stylesheet"/>
	    <link type="text/css" href=/blog_article/"/style/style.css" rel="stylesheet"/>