cup.util package

description:

utilization related module

cup.util.check_not_none(param)[source]

deprecated. Recommand using misc.check_not_none

cup.util.check_type(param, expect)[source]

deprecated. Recommand using misc.check_type in cup.util

Submodules

cup.util.conf module

description:

Complex and constructive conf support

class cup.util.conf.Configure2Dict(configure_file, remove_comments=True, separator=':')[source]

Bases: object

Configure2Dict support conf features below:

  1. comments

    As we support access/modify comments in a conf file, you should obey rules below:

    Comment closely above the object you want to comment. Do NOT comment after the line.

    Otherwise, you might get/set a wrong comment above the object.

  2. sections

    2.1 global section

    • if key:value is not under any [section], it is under the global layerby default

    • global section is the 0th layer section

    e.g.

    # test.conf
global-key: value
global-key1: value1

    2.2 child section

    • [section1] means a child section under Global. And it’s the 1st layer section

    • [.section2] means a child section under the nearest section above. It’s the 2nd layer section.

    • [..section3] means a child section under the nearest section above. And the prefix .. means it is the 3rd layer section

    e.g.: test.conf:

    global-key: value
[section]
    host:  abc.com
    port:  8080
    [.section_child]
        child_key: child_value
        [..section_child_child]
            control: ssh
            [...section_child_child_child]
                wow_key:  wow_value

    2.3 section access method

    get_dict method will convert conf into a ConfDict which is derived from python dict.

    • Access the section with confdict[‘section’][‘section-child’].

    • Access the section with confdict.get_ex(‘section’) with (value, comments)

  3. key:value and key:value array

    3.1 key:value

    key:value can be set under Global section which is closely after the 1st line with no [section] above.

    key:value can also be set under sections.

    # test.conf
key1: value1
[section]
    key_section: value_in_section
    [.seciton]
        key_section_child: value_section_child

    3.2 key:value arrays

    key:value arrays can be access with confdict[‘section’][‘disk’]. You will get a ConfList derived from python list.

    # test.conf
# Global layer, key:value
host: abc.com
port: 12345
# 1st layer [monitor]
@disk: /home/data0
@disk: /home/data1
[section]
    @disk: /home/disk/disk1
    @disk: /home/disk/disk2

  4. Example

# test.conf
# Global layer, key:value
host: abc.com
port: 12345
# 1st layer [monitor]
@disk: /home/data0
@disk: /home/data1
[section]
    @disk: /home/disk/disk1
    @disk: /home/disk/disk2

[monitor]
    timeout: 100
    regex:  sshd
    # 2nd layer that belongs to [monitor]
    [.timeout]
        # key:value in timeout
        max: 100
        # 3rd layer that belongs to [monitor] [timeout]
        [..handler]
            default: exit

# codes represent accessing conf and modifying conf
from __future__ import print_function
from cup.util import conf
# 1. read from test.conf
confdict = conf.Configure2Dict('test.conf', separator=':').get_dict()
print(confdict['host'])
print(confdict['port'])
print(confdict['section']['disk'])
print(confdict['monitor']['timeout']['handler'])
# 2. Change something and write back to test.conf
confdict['port'] = '10085'
confdict['monitor']['timeout']['handler']['default'] = 'stop'
confobj = conf.Dict2Configure(confdict, separator=':')
confobj.write_conf('./test.conf.new')
get_dict(ignore_error=False, encoding='utf8')[source]

get conf_dict which you can use to access conf info.

Parameters:

  • ignore_error – True, CUP will parse the conf file without catching exceptions

  • encoding – utf8 by default, you can change it to your encoding

class cup.util.conf.Dict2Configure(conf_dict, separator=':')[source]

Bases: object

Convert Dict into Configure. You can convert a ConfDict or python dict into a conf file.

set_dict(conf_dict)[source]

set a new conf_dict

write_conf(conf_file, encoding='utf8')[source]

write the conf into of the dict into a conf_file

Parameters:

  • conf_file – the file which will be override

  • encoding – ‘utf8’ by default, specify yours if needed

class cup.util.conf.HdfsXmlConf(filepath)[source]

Bases: object

hdfs xmlconf modifier.

Example:

# modify and write new conf into hadoop-site.xmlconf
xmlobj = xmlconf.HdfsXmlConf(xmlfile)

# get hdfs conf items into a python dict
key_values = xmlobj.get_items()

# modify hdfs conf items
for name in self._confdict['journalnode']['hadoop_site']:
    if name in key_values:
        key_values[name]['value'] =                 self._confdict['journalnode']['hadoop_site'][name]
else:
    key_values[name] = {
        'value': self._confdict['journalnode']['hadoop_site'][name],
        'description': ' '
    }
hosts = ','.join(self._confdict['journalnode']['host'])
key_values['dfs.journalnode.hosts'] = {
    'value': hosts, 'description':' journalnode hosts'
}

# write back conf items with new values
xmlobj.write_conf(key_values)
get_items()[source]

return hadoop config items as a dict.

Returns:

{
    'dfs.datanode.max.xcievers':  {
        'value': 'true', 'description': 'xxxxxxxxxx'
    },
    ......
}

write_conf(kvs, encoding='utf8')[source]

update config items with a dict kvs. Refer to the example above.

Parameters:

kvs – 

{
    key : { 'value': value, 'description': 'description'},
    ......
}

cup.util.constants module

description:

constant related module test-case-name: twisted.python.test.test_constants Copyright (c) Twisted Matrix Laboratories. See LICENSE for details.

license:

CUP dev team modified and redistrbuted this module, according to MIT license (http://opensource.org/licenses/mit-license.php) that Twisted lib obeys.

If any concern, plz contact mythmgn@gmail.com.

Mit License applied for twisted

http://www.opensource.org/licenses/mit-license.php Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

class cup.util.constants.FlagConstant(value=None)[source]

Bases: _Constant

L{FlagConstant} defines an attribute to be a flag constant within a collection defined by a L{Flags} subclass.

L{FlagConstant} is only for use in the definition of L{Flags} subclasses. Do not instantiate L{FlagConstant} elsewhere and do not subclass it.

class cup.util.constants.Flags[source]

Bases: Values

A L{Flags} subclass contains constants which can be combined using the common bitwise operators (C{|}, C{&}, etc) similar to a I{bitvector} from a language like C.

class cup.util.constants.NamedConstant[source]

Bases: _Constant

L{NamedConstant} defines an attribute to be a named constant within a collection defined by a L{Names} subclass.

L{NamedConstant} is only for use in the definition of L{Names} subclasses. Do not instantiate L{NamedConstant} elsewhere and do not subclass it.

class cup.util.constants.Names[source]

Bases: _ConstantsContainer

A L{Names} subclass contains constants which differ only in their names and identities.

class cup.util.constants.ValueConstant(value)[source]

Bases: _Constant

L{ValueConstant} defines an attribute to be a named constant within a collection defined by a L{Values} subclass.

L{ValueConstant} is only for use in the definition of L{Values} subclasses. Do not instantiate L{ValueConstant} elsewhere and do not subclass it.

class cup.util.constants.Values[source]

Bases: _ConstantsContainer

A L{Values} subclass contains constants which are associated with arbitrary values.

classmethod lookupByValue(value)[source]

Retrieve a constant by its value or raise a C{ValueError} if there is no constant associated with that value.

Parameters:

value – The value of one of the constants defined by C{cls}.

Raises:

ValueError – If C{value} is not the value of one of the constants defined by C{cls}.

Returns:

The L{ValueConstant} associated with C{value}.

cup.util.context module

description:

context for threadpool

class cup.util.context.ContextManager[source]

Bases: object

context for function call stack

call_with_context(new_context, func, *args, **kwargs)[source]

context is a {}

get_context(key)[source]

get the context that has the key

class cup.util.context.ContextTracker4Thread[source]

Bases: object

thread switch tracker

call_with_context(context, func, *args, **kwargs)[source]

call [func] and set up a context with it

current_context()[source]

get current context

get_context(key)[source]

get the context by key

cup.util.misc module

misc classes for internal use

class cup.util.misc.CAck(binit=False)[source]

Bases: object

ack class

getack_infobool()[source]

get bool info

setack_infobool(binit=False)[source]

set bool info

cup.util.misc.check_not_none(param)[source]

check param is not None

Raise:

NameError if param is None

cup.util.misc.check_type(param, expect)[source]

check type of the param is as the same as expect’s

Raise:

raise TypeError if it’s not the same

cup.util.misc.get_filename(backstep=0)[source]

Get the file name of the current code line.

Parameters:

backstep – will go backward (one layer) from the current function call stack

cup.util.misc.get_funcname(backstep=0)[source]

get funcname of the current code line

Parameters:

backstep – will go backward (one layer) from the current function call stack

cup.util.misc.get_lineno(backstep=0)[source]

Get the line number of the current code line

Parameters:

backstep – will go backward (one layer) from the current function call stack

cup.util.typechecker module

description:

check type and return True or False

cup.util.typechecker.check_not_none(param)[source]

check param is not None

Raise:

NameError if param is None

cup.util.typechecker.check_type(param, expect)[source]

check type of the param is as the same as expect’s

Raise:

raise TypeError if it’s not the same

cup.util.typechecker.raise_error(param, expect_type)[source]

raise type error if the type(param) != expect_type

Raise:

TypeError

cup.util.typechecker.raise_ifnone(param)[source]

raise NameError if param is None