cup.util package

Submodules

cup.util.conf module

description:Complex and constructive conf support
class cup.util.conf.Configure2Dict(configure_file, remove_comments=True, separator=':')[source]

Bases: object

Configure2Dict support conf features below:

  1. comments

    As we support access/modify comments in a conf file, you should obey rules below:

    Comment closely above the object you want to comment. Do NOT comment after the line.

    Otherwise, you might get/set a wrong comment above the object.

  2. sections

    2.1 global section

    • if key:value is not under any [section], it is under the global layer
      by default
    • global section is the 0th layer section

    e.g. test.conf:

    # test.conf global-key: value global-key1: value1

    2.2 child section
    • [section1] means a child section under Global. And it’s the
      1st layer section
    • [.section2] means a child section under the nearest section
      above. It’s the 2nd layer section.
    • [..section3] means a child section under the nearest section
      above. And the prefix .. means it is the 3rd layer section

    e.g.: test.conf:

    global-key: value
    [section]
        host:  abc.com
        port:  8080
        [.section_child]
            child_key: child_value
            [..section_child_child]
                control: ssh
                [...section_child_child_child]
                    wow_key:  wow_value
    
    2.3 section access method

    get_dict method will convert conf into a ConfDict which is derived

    from python dict.

    • Access the section with confdict[‘section’][‘section-child’].
    • Access the section with confdict.get_ex(‘section’) with (value,
      comments)
  3. key:value and key:value array

    3.1 key:value

    key:value can be set under Global section which is closely after the 1st line with no [section] above.

    key:value can also be set under sections.

    # test.conf
    key1: value1
    [section]
        key_section: value_in_section
        [.seciton]
            key_section_child: value_section_child
    
    3.2 key:value arrays

    key:value arrays can be access with confdict[‘section’][‘disk’]. You will get a ConfList derived from python list.

    # test.conf
    # Global layer, key:value
    host: abc.com
    port: 12345
    # 1st layer [monitor]
    @disk: /home/data0
    @disk: /home/data1
    [section]
        @disk: /home/disk/disk1
        @disk: /home/disk/disk2
    

4. Example

# test.conf
# Global layer, key:value
host: abc.com
port: 12345
# 1st layer [monitor]
@disk: /home/data0
@disk: /home/data1
[section]
    @disk: /home/disk/disk1
    @disk: /home/disk/disk2

[monitor]
    timeout: 100
    regex:  sshd
    # 2nd layer that belongs to [monitor]
    [.timeout]
        # key:value in timeout
        max: 100
        # 3rd layer that belongs to [monitor] [timeout]
        [..handler]
            default: exit
get_dict(ignore_error=False)[source]

get conf_dict which you can use to access conf info.

Parameters:ignore_error – True, CUP will parse the conf file without catching exceptions
class cup.util.conf.Dict2Configure(conf_dict, separator=':')[source]

Bases: object

Convert Dict into Configure. You can convert a ConfDict or python dict into a conf file.

set_dict(conf_dict)[source]

set a new conf_dict

write_conf(conf_file)[source]

write the conf into of the dict into a conf_file

class cup.util.conf.HdfsXmlConf(filepath)[source]

Bases: object

hdfs xmlconf modifier.

Example:

# modify and write new conf into hadoop-site.xmlconf
xmlobj = xmlconf.HdfsXmlConf(xmlfile)

# get hdfs conf items into a python dict
key_values = xmlobj.get_items()

# modify hdfs conf items
for name in self._confdict['journalnode']['hadoop_site']:
    if name in key_values:
        key_values[name]['value'] =                 self._confdict['journalnode']['hadoop_site'][name]
else:
    key_values[name] = {
        'value': self._confdict['journalnode']['hadoop_site'][name],
        'description': ' '
    }
hosts = ','.join(self._confdict['journalnode']['host'])
key_values['dfs.journalnode.hosts'] = {
    'value': hosts, 'description':' journalnode hosts'
}

# write back conf items with new values
xmlobj.write_conf(key_values)
get_items()[source]

return hadoop config items as a dict.

::
{
‘dfs.datanode.max.xcievers’: {
‘value’: ‘true’, ‘description’: ‘xxxxxxxxxx’

}

write_conf(kvs)[source]

update config items with a dict kvs. Refer to the example above.

::
{

}

cup.util.constants module

description:

constant related module test-case-name: twisted.python.test.test_constants Copyright (c) Twisted Matrix Laboratories. See LICENSE for details.

license:
CUP dev team modified and redistrbuted this module,

according to MIT license (http://opensource.org/licenses/mit-license.php) that Twisted lib obeys.

If any concern, plz contact mythmgn@gmail.com.

Mit License applied for twisted

http://www.opensource.org/licenses/mit-license.php Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

class cup.util.constants.NamedConstant[source]

Bases: cup.util.constants._Constant

L{NamedConstant} defines an attribute to be a named constant within a collection defined by a L{Names} subclass.

L{NamedConstant} is only for use in the definition of L{Names} subclasses. Do not instantiate L{NamedConstant} elsewhere and do not subclass it.

class cup.util.constants.ValueConstant(value)[source]

Bases: cup.util.constants._Constant

L{ValueConstant} defines an attribute to be a named constant within a collection defined by a L{Values} subclass.

L{ValueConstant} is only for use in the definition of L{Values} subclasses. Do not instantiate L{ValueConstant} elsewhere and do not subclass it.

class cup.util.constants.FlagConstant(value=None)[source]

Bases: cup.util.constants._Constant

L{FlagConstant} defines an attribute to be a flag constant within a collection defined by a L{Flags} subclass.

L{FlagConstant} is only for use in the definition of L{Flags} subclasses. Do not instantiate L{FlagConstant} elsewhere and do not subclass it.

class cup.util.constants.Names[source]

Bases: cup.util.constants._ConstantsContainer

A L{Names} subclass contains constants which differ only in their names and identities.

class cup.util.constants.Values[source]

Bases: cup.util.constants._ConstantsContainer

A L{Values} subclass contains constants which are associated with arbitrary values.

classmethod lookupByValue(value)[source]

Retrieve a constant by its value or raise a C{ValueError} if there is no constant associated with that value.

@param value: The value of one of the constants defined by C{cls}.

@raise ValueError: If C{value} is not the value of one of the constants
defined by C{cls}.

@return: The L{ValueConstant} associated with C{value}.

class cup.util.constants.Flags[source]

Bases: cup.util.constants.Values

A L{Flags} subclass contains constants which can be combined using the common bitwise operators (C{|}, C{&}, etc) similar to a I{bitvector} from a language like C.

cup.util.context module

description:context for threadpool
class cup.util.context.ContextManager[source]

Bases: object

context for function call stack

call_with_context(new_context, func, *args, **kwargs)[source]

context is a {}

get_context(key)[source]

get the context that has the key

class cup.util.context.ContextTracker4Thread[source]

Bases: object

thread switch tracker

call_with_context(context, func, *args, **kwargs)[source]

call [func] and set up a context with it

current_context()[source]

get current context

get_context(key)[source]

get the context by key

cup.util.misc module

misc classes for internal use

class cup.util.misc.CAck(binit=False)[source]

Bases: object

ack class

getack_infobool()[source]

get bool info

setack_infobool(binit=False)[source]

set bool info

cup.util.misc.check_not_none(param)[source]

check param is not None

Raise:NameError if param is None
cup.util.misc.check_type(param, expect)[source]

check type of the param is as the same as expect’s

Raise:raise TypeError if it’s not the same
cup.util.misc.get_filename(backstep=0)[source]

Get the file name of the current code line.

Parameters:backstep – will go backward (one layer) from the current function call stack
cup.util.misc.get_funcname(backstep=0)[source]

get funcname of the current code line

Parameters:backstep – will go backward (one layer) from the current function call stack
cup.util.misc.get_lineno(backstep=0)[source]

Get the line number of the current code line

Parameters:backstep – will go backward (one layer) from the current function call stack

Module contents

description:utilization related module
cup.util.check_not_none(param)[source]

deprecated. Recommand using misc.check_not_none

cup.util.check_type(param, expect)[source]

deprecated. Recommand using misc.check_type in cup.util