Copy&Waste

Posts tagged "python"

Searching Your Corporate LDAP/AD with python

There are a ton of examples out there on how to utilize ldap in python. I recently worked on a simple project which required active directory authentication using ldap. The following is simple script I've used hundreds of times.

 
#!/usr/bin/env python 
#binds to ldap, queries for a specific AD account 
import ldap 

def Search(server,port, auth_user, auth_pass, ldap_user,attrs):
     #base_dn should reflect your domain 
     base_dn="dc=yourcompanydomain,dc=com" 
     found_results="" 
     l = ldap.initialize('ldap://%s:%s' % (server, port) ) 
     l.simple_bind_s(auth_user, auth_pass) 
     try: 
        search_result= l.search(base_dn,ldap.SCOPE_SUBTREE,'sAMAccountName='+ldap_user,attrs) 
        result_set =[] 
        while 1: 
           result_type, result_data = l.result(search_result,0) 
           if (result_data ==[]):
              break 
           else:
           if result_type == ldap.RES_SEARCH_ENTRY: 
              result_set.append(result_data) 
              print len(result_set) 
              for line in result_set: 
                 print line 
     except ldap.LDAPError, e: 
        print e 
     results = Search("192.168.5.10",3268,"akonkol@yourcompanydomain.com","mysecretpassword","akonkol",['givenName','sn','mail']) 

This will return the attributes you supplied (givenName, sn, mail)

Tagged as ldap , python
Written by Andrew Konkol on November 18th, 2009 | 0 Comments

Pexpect, Python, and Managing Devices -- Tratto

tratto-logo A few months ago I decided to write a web application that would essentially run like RANCID, I named it "tratto." Since then I decided that it would be better to nail down a basic python framework first, then integrate it into say.. a django application. In all of my years as an engineer I had never leveraged expect to accomplish simple and repetitive tasks. I am a recent python "convert" and wanted to write a simple app that could be used as a framework for managing and monitoring network connected devices and hosts. Tratto uses pexpectto connect and parse ssh and telnet sessions. This framework provides an easy way to connect to remote devices and issue commands and store the output. I also wanted an easy way to "extend" this framework and be able to add ways to connect to any operating system (or at least use default shell behavior as a baseline). I manage a wide variety of devices and I wanted to support at least the default implementations of Cisco IOS, OpenBSD, Mac OS X, and Aruba OS. Here is how you would add an operating system's parameters to Tratto (Systems.py):

class ArubaOS(OperatingSystem):
	'''aruba configs'''
	PROMPTLINE	='#'
	PAGINATES 	=True
	DISABLE_PAGINATION = 'terminal length 0'
	GET_CONFIG	="show run"

There are 3 files included in Tratto right now:

  1. Connectivity.py -- This is a class which manages sessions using pexpect
  2. Systems.py -- This is the class which manages all the operating parameters
  3. driver.py -- This is an example file of how to use Tratto to fetch whatever info you want

Here is an example of how to use the framework to connect to devices and issue commands:

#!/usr/bin/env python

import Connectivity
import Systems

#telnet to a cisco switch
m = Systems.OperatingSystems['IOS']
s = Connectivity.Session("192.168.6.1",23,telnet,m)
s.login("akonkol", "mypass")
s.sendcommand("show ver")
s.sendcommand("show clock")
s.sendcommand("show run")
s.logout()

#ssh to a apple machine
m = Systems.OperatingSystems['OSX']
s = Connectivity.Session("127.0.0.1",22,"ssh",m)
s.login("akonkol", "mypass")
#sendcommand will echo response by default, you can store that
#response in a variable if you wish
result = s.sendcommand("df -h")
print result
s.getversion()
s.logout()

#ssh to openbsd box
m = Systems.OperatingSystems['OBSD']
s = Connectivity.Session("192.168.5.1",22,"ssh",m)
s.login("akonkol", "mypass")
print s.sendcommand("cat /etc/passwd")
print s.sendcommand("arp -a")
s.logout()

The Future

With Tratto you can technically pull information from any networked device and use that data for whatever you please. Current ideas are integrating Tratto into

  • a config repository application with a web frontend (like RANCID)
  • a network mapping application using cdp neighbors
  • monitoring platform which performs different commands based on certain scenarios ("show interfaces" when IP SLA shows latency)

This is my first attempt at releasing python software, so if you think something could be better let me know. Download Tratto

Tagged as cisco , pexpect python , ssh telnet
Written by Andrew Konkol on January 26th, 2010 | 4 Comments

Megamillions, nerd.

A lot of talk about the lottery today at lunch.. i've never bought a ticket.

#!/usr/bin/python
import random

for i in range(1,6):
        print random.randrange(1,56)


print random.randrange(1,46)

Tagged as lottery , python
Written by Andrew Konkol on January 4th, 2011 | 0 Comments

Python, Django, Open Flash Chart (pyofc2)

Graphs are a pain in the ass. I first attempted to get PYOFC2 to work about a year ago and gave up, I just didn't care enough. Now I want graphs, and have figured out how to use it (at least for a pie chart). urls.py

from django.conf.urls.defaults import *
from django.conf import settings

# the chart data views
urlpatterns = patterns('yourapp.views',
    ('^data/$','chart_data_two'),
)

urlpatterns += patterns('django.views.generic.simple',
    (r'^$', 'direct_to_template', {'template': 'index.html'}),
)

templates/index.html

Chart!








<h1>Pie Chart!</h1>
<div id="chart_demo" class="chart"> </div>

ls -l static/*

-rwxr-xr-x 1 akonkol akonkol    773 2011-03-02 16:08 expressInstall.swf
-rw-r--r-- 1 akonkol akonkol 263109 2011-03-02 16:08 open-flash-chart.swf
-rw-r--r-- 1 akonkol akonkol   9759 2011-03-02 16:08 swfobject.js

yourapp/views.py

from django.http import HttpResponse
from pyofc2  import *
import random
import time

def chart_data_two(request):
    t = title(text='Crapface Graph')
    p1 = pie()
    values = [ pie_value(label="crap", value=4), pie_value(label="face", value=20)]
    p1.values = values
    chart = open_flash_chart()
    chart.title = t
    chart.add_element(p1)
    return HttpResponse(chart.render())

Produces: crap_face References demodjofc2_demo

Tagged as charts , django graphing , python
Written by Andrew Konkol on March 3rd, 2011 | 0 Comments

Hatch - Create and Share Configuration Templates

About a month ago I wrote what I call hatch, which helps you create templates and generate configurations on the fly. Originally the idea was to allow people to store configuration files, but I nixed that idea and it reduced my amount of code by about 50%. Instead I added a few convenient features such as a copy-to-clipboard button, comments, markdown syntax, tagging, and user profiles with avatar support. On top of all the new features, hatch went through a re-design: To make hatch better, I've started eating my own dog food and so far so good.

Tagged as hatch , python templates
Written by Andrew Konkol on July 18th, 2011 | 0 Comments

Unique Slugs for Django Objects

I'm working on a new project and decided to use slugs to access object information.  The built-in in "slugify" does not generate unique slugs for objects, but I found a solution. Overriding the save method allows us to check to see if there are any other objects which have the same slug such as "how-to-make-a-table" and if there are we append a number to it such as "how-to-make-a-table-2." Below is a real-world example of a model "Project" which has pretty typical fields in it and the code to generate a unique slug. file: /myproject/myapp/models.py

from django.contrib.auth.models import User
from sorl.thumbnail import ImageField
import datetime
import re

class Project(models.Model):
        title = models.CharField(max_length=255,blank=True)
        description = models.TextField(blank=True)
        photo = ImageField(upload_to=get_project_upload_path, blank=True, null=True)
        author = models.ForeignKey(User, editable=False,blank=True, null=True)
        pub_date = models.DateTimeField(editable=False, blank=True)
        slug = models.SlugField(unique=True,blank=True)

        def save(self, *args, **kwargs):
		#set pub_date as right now
                self.pub_date=datetime.datetime.now()
		
		#As long as this object does NOT have a slug
                if not self.slug:
                   from django.template.defaultfilters import slugify
		
		   #Take the title and replace spaces with hypens, make lowercase
                   potential_slug = slugify(self.title)
                   self.slug = potential_slug

                   while True:
                      try:
			 #try to save the object
                         super(Project, self).save(*args, **kwargs)
		      
		      #if this slug already exists we get an error
                      except IntegrityError:
			 #match the slug or look for a trailing number
                         match_obj = re.match(r'^(.*)-(\d+)$', self.slug)
			 
			 #if we find a match
                         if match_obj:
			    #take the found number and increment it by 1
                            next_int = int(match_obj.group(2)) + 1
                            self.slug = match_obj.group(1) + "-" + str(next_int)
                         else:
			    #There are no matches for -# so create one with -2
                            self.slug += '-2'
		      #different error than IntegrityError
                      else:
                         break

        def __unicode__(self):
                return self.title
Tagged as django , python slugs
Written by Andrew Konkol on July 20th, 2011 | 0 Comments

Compiling Netsnmp for Ubuntu with Python Bindings

apt-get install gcc apt-get install libperl-dev apt-get install python2.7-dev Download the netsnmp source: wget http://downloads.sourceforge.net/project/net-snmp/net-snmp/5.7.1/net-snmp-5.7.1.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fnet-snmp%2Ffiles%2Fnet-snmp%2F5.7.1%2F&ts=1318446955&use_mirror=surfnet tar xzvf net-snmp-5.7.1.tar.gz cd net-snmp-5.7.1 ./configure --with-python-modules Answer the questions make make install ldconfig akonkol@cmg005:/home/akonkol/net-snmp-5.7.1# python Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) [GCC 4.5.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import netsnmp >>>

Tagged as python , snmp
Written by Andrew Konkol on October 12th, 2011 | 1 Comments

Multiprocessing SNMP with Python

I've written a ton of snmp monitoring scripts and they all suck because they are blocking and take "too long" to return results for a large amount of hosts. So how would we make this process faster and make us happier?

multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.

IBM has published a wonderful article on Multiprocessing with Python in which I have modified their snippets for my purposes below. To leverage NetSNMP in python you need to compile it with python modules, I wrote a post about this yesterday in Compiling Netsnmp for Ubuntu. After battling through the compiling bit you should be able to get the example in the IBM post to work. I modified this example *slightly* to accommodate instance identifier (iid).  Hopefully this will be suitable for a new project I'm working on which polls thousands of snmp oids to measure latency between routers using the concept I posted on hatch - Configure RTR/SLA Ping in IOS template. 

!/usr/bin/env python
"""
This is a multiprocessing wrapper for Net-SNMP.
This makes a synchronous API asynchronous by combining
it with Python2.6
"""

import netsnmp
from multiprocessing import Process, Queue, current_process

class HostRecord():
    """This creates a host record"""
    def __init__(self,
                 hostname = None,
                 query = None):
        self.hostname = hostname
        self.query = query

class SnmpSession():
    """A SNMP Session"""
    def __init__(self,
                oid = "sysDescr",
                iid="0",
                Version = 2,
                DestHost = "localhost",
                Community = "public",
                Verbose = True,
                ):
        self.oid = oid
        self.Version = Version
        self.DestHost = DestHost
        self.Community = Community
        self.Verbose = Verbose
        self.var = netsnmp.Varbind(oid, iid)
        self.hostrec = HostRecord()
        self.hostrec.hostname = self.DestHost

    def query(self):
        """Creates SNMP query

        Fills out a Host Object and returns result
        """
        try:
            result = netsnmp.snmpget(self.var,
                                Version = self.Version,
                                DestHost = self.DestHost,
                                Community = self.Community)
            self.hostrec.query = result
        except Exception, err:
            if self.Verbose:
                print err
            self.hostrec.query = None
        finally:
            return self.hostrec

def make_query(host):
    """This does the actual snmp query

    This is a bit fancy as it accepts both instances
    of SnmpSession and host/ip addresses.  This
    allows a user to customize mass queries with
    subsets of different hostnames and community strings
    """
    if isinstance(host,SnmpSession):
        return host.query()
    else:
        s = SnmpSession(DestHost=host)
        return s.query()

# Function run by worker processes
def worker(input, output):
    for func in iter(input.get, 'STOP'):
        result = make_query(func)
        output.put(result)

def main():
    """Runs everything"""

    #clients
    hosts = [
    SnmpSession(DestHost="10.71.1.1", Community="my-pub-community", oid="1.3.6.1.4.1.9.9.42.1.2.10.1.1", iid="1"),
    SnmpSession(DestHost="10.81.1.1", Community="my-pub-community", oid="1.3.6.1.4.1.9.9.42.1.2.10.1.1", iid="123")
    ]
    NUMBER_OF_PROCESSES = len(hosts)

    # Create queues
    task_queue = Queue()
    done_queue = Queue()

    #submit tasks
    for host in hosts:
        task_queue.put(host)

    #Start worker processes
    for i in range(NUMBER_OF_PROCESSES):
        Process(target=worker, args=(task_queue, done_queue)).start()

     # Get and print results
    print 'Unordered results:'
    for i in range(len(hosts)):
        print '\t', done_queue.get().query

    # Tell child processes to stop
    for i in range(NUMBER_OF_PROCESSES):
        task_queue.put('STOP')
        #print "Stopping Process #%s" % i

if __name__ == "__main__":
    main()
Tagged as python , snmp
Written by Andrew Konkol on October 13th, 2011 | 0 Comments

Tratto London (Revision): Escalate Privileges

Two years ago I wrote a script that you could use to send commands to telnet/ssh enabled devices called tratto.  Since then I have changed jobs and with new jobs comes new enviornments.  I can no longer send commands like "show run" without sending the enable command (enable is like the 'su' of unix).  I re-visited my code... which is always an entertaining.  Anyway, I added an "escalateprivileges" command and added a string to the Systems object so you can store what the escalation command is for different operating systems.

You can download it here or via github

Connectivity.py

        def escalateprivileges(self, escalated_password=None):
                escalated_password = escalated_password
                if self.connected:
                        self.connection.sendline(self.operatingsystem.ESCALATE_COMMAND)
                        i = self.connection.expect(r"(?i)password[\s:]+")
                        if i==0:
                                self.connection.sendline(escalated_password)
                                i = self.connection.expect(self.operatingsystem.PROMPTLINE)
                                if i==0:
                                        if("denied" in self.connection.before):
                                           print "***Escalation FAILED***"
                                           print self.connection.before
                                        else:
                                           print "***Escalation Successful***"
                else:
                        raise SessionError("***Not Connected***")

Systems.py

class OperatingSystem(object):
        ESCALATE_COMMAND=''
        PAGINATES       =False
        VERSION         =''
        PROMPTLINE      =''

..

class CiscoIOS(OperatingSystem):
        '''cisco ios'''
        PROMPTLINE      = r'[-\w]+[>#]'
        GET_CONFIG      ='show running-config'
        PAGINATES       =True
        VERSION         ='show version'
        DISABLE_PAGINATION = 'terminal length 0'
        ESCALATE_COMMAND='enable'
...

Example usage:

#!/usr/bin/env python

import Connectivity
import Systems

#telnet to a cisco switch

m = Systems.OperatingSystems['IOS']
s = Connectivity.Session("10.10.1.1",23,"telnet",m)
s.login("akonkol", "mypassword")
s.escalateprivileges('myenablepassword')
#s.sendcommand("show ver")
s.sendcommand("show clock")
s.sendcommand("show run")
s.sendcommand("show start")
s.logout()
Tagged as cisco , pexpect python , ssh telnet , tratto
Written by Andrew Konkol on April 16th, 2012 | 0 Comments

Creating your own Exceptions in Python

I needed to handle errors better so I looked into creating my own exceptions.  It easy, and this is a dead simple example.

#!/usr/bin/python

class AdditionError(Exception):
        pass

x = 2
y = 'i'

try:
   print  x + y

except:
   raise AdditionError("Couldn't add x + y ")
    
 
root@echo:/home/akonkol/Code/exception# ./addition.py 
Traceback (most recent call last): 
  File "./addition.py", line 14, in 
     raise AdditionError("Couldn't add x + y ") 
__main__.AdditionError: Couldn't add x + y
Tagged as exception , python
Written by Andrew Konkol on June 20th, 2012 | 0 Comments

Converting strings escaped string literals or raw string

I recenly wrote a django app which needed to store regex strings in a model.  Since I couldn't find a way to store raw strings in models I wrote a function to convert unicode or string into string literals.

def to_raw_string(s):
        if isinstance(s,str):
           s = s.encode('string-escape')
        if isinstance(s,unicode):
           s = s.encode('unicode-escape')
        return s

Tagged as django , python raw string , rawstring string , unicode
Written by Andrew Konkol on July 24th, 2012 | 0 Comments

Generate IP Addresses and valid CIDR notations

I needed to test some networking tools I've been developing, and to do that I needed IP addresses. To get IP addresses I wrote this snippet.

#!/usr/bin/python

import random
def generate_ip():
   CLASSES='ABC'
   ip_class = random.choice(CLASSES)

   if ip_class == "A":
        first_octet = random.randint(1,126)

   if ip_class == "B":
        first_octet = random.randint(128,191)

   if ip_class == "C":
        first_octet = random.randint(192,223)

   second_octet = random.randint(0,254)
   third_octet = random.randint(0,254)
   fourth_octet = random.randint(0,254)
   return "%i.%i.%i.%i" %(first_octet,second_octet,third_octet,fourth_octet)

I also needed to generate random valid slash notations for a given ip:

def generate_cidr(network_id):
   first_octet = int(str(network_id).split(".")[0])

   if first_octet <= 126 and first_octet >= 1:
      cidr_bits =  random.randint(8,30)

   if first_octet >=127 and first_octet <= 191:
      cidr_bits =  random.randint(16,30)

   if first_octet >=192 and first_octet <= 223:
      cidr_bits =  random.randint(24,30)

   return cidr_bits
Tagged as cidr , IP python
Written by Andrew Konkol on October 22nd, 2013 | 0 Comments

Generating Layer 3 Interface DNS records for cisco devices

Traceroute looks up the PTR dns records for each hop. A lot of providers create ptr records that describe the layer 3 interface of a given router:

akonkol@use:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  router1-dal.linode.com (67.18.7.161)  0.451 ms  0.568 ms  0.698 ms
 2  ae2.car01.dllstx2.networklayer.com (67.18.7.89)  0.184 ms  0.215 ms  24.803 ms
 3  po101.dsr01.dllstx2.networklayer.com (70.87.254.73)  0.667 ms  0.726 ms  1.026 ms
 4  po21.dsr01.dllstx3.networklayer.com (70.87.255.65)  0.793 ms  0.860 ms  0.911 ms
 5  ae16.bbr02.eq01.dal03.networklayer.com (173.192.18.228)  0.470 ms  0.451 ms  0.456 ms
 6  ae7.bbr01.eq01.dal03.networklayer.com (173.192.18.208)  0.440 ms  0.437 ms  0.417 ms
 7  50.97.16.37 (50.97.16.37)  0.486 ms  0.512 ms  0.491 ms
 8  72.14.233.85 (72.14.233.85)  0.555 ms 72.14.233.77 (72.14.233.77)  10.054 ms  10.044 ms
 9  64.233.175.148 (64.233.175.148)  7.835 ms 72.14.237.219 (72.14.237.219)  3.690 ms  3.676 ms
10  209.85.249.69 (209.85.249.69)  7.489 ms 72.14.237.123 (72.14.237.123)  7.450 ms 209.85.249.66 (209.85.249.66)  7.409 ms
11  216.239.46.39 (216.239.46.39)  7.366 ms  7.332 ms 216.239.46.63 (216.239.46.63)  7.357 ms
12  * * *
13  google-public-dns-a.google.com (8.8.8.8)  7.410 ms  7.436 ms  7.418 ms

Depending on how much equipment you manage you could have hundreds or even thousands of layer 3 interfaces. This problem intrigued me so I wrote a script that leverages tratto; a framework that I built back in 2012 to ssh/telnet to devices. The script is called l3toptr.py which gets layer 3 interface information and constructs a fqdn.

Connection Code

The following snippet shows a few things: setting up the ssh session, issuing a "show ip interface brief" storing the results in "ip_ints."

#connection setup
import Connectiviy, Systems

device = args['device']
username = args['username']
os_type = Systems.OperatingSystems['IOS']


session = Connectivity.Session(device,port,transport,os_type)
session.login(username, password)
ip_ints = session.sendcommand("sho ip int br")
session.logout()

Parsing Code

We take ip_ints and for each line that contains an ip address, we split the line by "whitespace" which allows us to access the interface name and number separately allowing us to assign these values to int_name and int_ip. Based on the interface name we come up with an abbreviation: GE for gigabit interfaces, FE for fast ethernet interfaces,etc... Since DNS records do now allow for slashes, we replace those with hyphens and assign it to clean_suffix.

#for each line of sho ip int brief

for line in ip_ints.split('\r\n'):
   contains_ip = re.findall( r'[0-9]+(?:\.[0-9]+){3}', line )
   if contains_ip:
        line_chunks = line.split()

        #[0]= gigabitethernet1/2
        int_name = line_chunks[0]

        #[1] = 10.a.b.c
        int_ip = line_chunks[1]

        if "GigabitEthernet" in int_name:
           prefix = "GE"
           suffix = int_name[15:]

        if "FastEthernet" in int_name:
           prefix = "FE"
           suffix = int_name[12:]

        if "Loopback" in int_name:
           prefix ="LO"
           suffix = int_name[8:]

        if "Vlan" in int_name:
           prefix="VL"
           suffix=int_name[4:]

        if "Tunnel" in int_name:
           prefix="TU"
           suffix= int_name[6:]

        #ignore NVIs
        if "NVI" in int_name:
          break

        #replace interface number slash with hyphen
        clean_suffix = re.sub('\/','-',suffix)

Schema and Return Code

The remaining code checks for any user defined schema. There are three variables you can specify in this "schema" (or format string): interface_name, interface_number, hostname, and ip_address. You would specify something like this to use a custom fqdn format: ./l3toPTR.py -u akonkol -d 10.24.2.1 -s "interface_name.hostname.mycompany.com" and interface_name and hostname would be replaced with the dynamically learned information for device living at 10.24.2.1.

if args['schema']:
           schema = args['schema']
           dns_line = schema.replace('interface_name',prefix)
           dns_line = dns_line.replace('interface_number',clean_suffix)
           dns_line = dns_line.replace('hostname',hostname)
           dns_line = dns_line.replace('ip_address',int_ip)
           print dns_line
        else:
           print prefix + clean_suffix  + "." + hostname +  domainname + "," +  int_ip

Some Examples:

akonkol@echo:~/Code$ ./l3toPTR.py -u akonkol -d 10.45.1.2
Password:
GE0-0.chicagorouter02.network.exampleco.com,10.45.1.2
GE0-1.chicagorouter02.network.exampleco.com,24.140.215.81
LO10.chicagorouter02.network.exampleco.com,10.0.45.2
LO99.chicagorouter02.network.exampleco.com,24.140.215.82



akonkol@echo:~/Code$ ./l3toPTR.py -u akonkol -d 10.45.1.2 -dn konkol.com
Password:
GE0-0.chicagorouter02.konkol.com,10.45.1.2
GE0-1.chicagorouter02.konkol.com,24.140.215.81
LO10.chicagorouter02.konkol.com,10.0.45.2
LO99.chicagorouter02.konkol.com,24.140.215.82



akonkol@echo:~/Code$ ./l3toPTR.py -u akonkol -d 10.45.1.2 -s "interface_name-interface_number.mycompany.com,ip_address"
Password:
GE-0-0.mycompany.com,10.45.1.2
GE-0-1.mycompany.com,24.140.215.81
LO-10.mycompany.com,10.0.45.2
LO-99.mycompany.com,24.140.215.82

Wait a second, this isn't actually creating the DNS entries! You would be correct, and I'm willing to bet that if you've read this far that you can figure how to do that... There are a few ways to do this and it all depends on what you are using for your DNS server (bind, windows, etc). The above script generates a CSV output which you could loop through and use to create dns records. Here is a great article on creating ptr records in bulk on a windows dns server DNS Bulk PTR records creation.

Tagged as cisco , dns ios , ptr python , tratto
Written by Andrew Konkol on July 22nd, 2014 | 0 Comments
Copyright © 2022 Andrew Konkol | Contact | Feeds