Python Connection

💪 Difficulty: advanced 🥷

⏳ Time to complete: as long as it takes to write your script

💬 Description

With the Custom Connector, you can write your own Python scripts and execute them. With time we’ll add different languages and a better interface.

🤑 Use Case

There are many use cases for scheduling your own Python script. Maybe you have a ML algorithm you wanna run at some specified frequency. Perhaps you want to trigger a POST call based on some trigger event in your database. Or you have an internal software system that you want to connect to. The possibilities are endless.

🛠️ Setup

You can check out the template for running Python scripts below but we do encourage scheduling a call with us to get you up and running quickly.

# You must include the packages in your script
import sys
import json
import sqlalchemy
from sqlalchemy import create_engine
import os
import pandas as pd
from datetime import datetime

# These allows us to store the output data of your script on the database you've connected to Less
dbconnection = json.loads(os.getenv('dbconnection'))

db_tables = {}
db_tables['event_type'] = "tables"
db_tables['db_tables'] = []
db_tables['db_id'] = dbconnection["id"]


# This create a connection to your existing database so we can store data their
engine = create_engine("mysql+pymysql://{user}:{pw}@{serverurl}:{port}/{db}"
                       .format(serverurl=dbconnection["server"],
                               user=dbconnection["username"],
                               pw=dbconnection["password"],
                               db=dbconnection["database"],
                               port="3306"),
                               connect_args={'ssl': {'requiressl': True}})


# The function that allows us to store your data (you MUST use this one below if you want to store data on Less) 
def storeData(data, tablename):
	  global db_tablesx
    db_tables['db_tables'].append(tablename)
    data.to_sql(tablename, con = engine, if_exists = "replace", chunksize = 1000)


# This allows Less to notice if your script fails so we can notice you on the platform
result = {}
result['startedAt'] = str(pd.to_datetime(datetime.now()))
result['event_type'] = "result"
result['status'] = '200'
result['records'] = 0

try:
	
############## INPUT YOUR SCRIPT BELOW HERE ###############

		
#Note that you must use the storeData function from above to store your data. The data variable must be a dataframe and the tablename can be anything you want. This is the name the table will have in the database

#EXAMPLE BELOW: 
    storeData(df, "testing_custom_connector")
    result["records"] += df.shape[0]


############## UNTIL HERE (can be as long as your want) ###############
    
# This is for reading the performance of your script
    result['from_date'] = ''
    result['to_date'] = ''
    result['stoppedAt'] = str(pd.to_datetime(datetime.now()))
    print(json.dumps(result))
    print(json.dumps(db_tables))

    
except Exception as e:
    #error out exeptions
    result['status'] = '500'
    result['records'] = 0
    result['stoppedAt'] = str(pd.to_datetime(datetime.now()))
    result['error'] = str(e)
    print(json.dumps(result))