⏳ Time to complete: as long as it takes to write your script
💬 Description
With the Custom Connector, you can write your own Python scripts and execute them. With time we’ll add different languages and a better interface.
🤑 Use Case
There are many use cases for scheduling your own Python script. Maybe you have a ML algorithm you wanna run at some specified frequency. Perhaps you want to trigger a POST call based on some trigger event in your database. Or you have an internal software system that you want to connect to. The possibilities are endless.
🛠️ Setup
You can check out the template for running Python scripts below but we do encourage scheduling a call with us to get you up and running quickly.
# You must include the packages in your script
import sys
import json
import sqlalchemy
from sqlalchemy import create_engine
import os
import pandas as pd
from datetime import datetime
# These allows us to store the output data of your script on the database you've connected to Less
dbconnection = json.loads(os.getenv('dbconnection'))
db_tables = {}
db_tables['event_type'] = "tables"
db_tables['db_tables'] = []
db_tables['db_id'] = dbconnection["id"]
# This create a connection to your existing database so we can store data their
engine = create_engine("mysql+pymysql://{user}:{pw}@{serverurl}:{port}/{db}"
.format(serverurl=dbconnection["server"],
user=dbconnection["username"],
pw=dbconnection["password"],
db=dbconnection["database"],
port="3306"),
connect_args={'ssl': {'requiressl': True}})
# The function that allows us to store your data (you MUST use this one below if you want to store data on Less)
def storeData(data, tablename):
global db_tablesx
db_tables['db_tables'].append(tablename)
data.to_sql(tablename, con = engine, if_exists = "replace", chunksize = 1000)
# This allows Less to notice if your script fails so we can notice you on the platform
result = {}
result['startedAt'] = str(pd.to_datetime(datetime.now()))
result['event_type'] = "result"
result['status'] = '200'
result['records'] = 0
try:
############## INPUT YOUR SCRIPT BELOW HERE ###############
#Note that you must use the storeData function from above to store your data. The data variable must be a dataframe and the tablename can be anything you want. This is the name the table will have in the database
#EXAMPLE BELOW:
storeData(df, "testing_custom_connector")
result["records"] += df.shape[0]
############## UNTIL HERE (can be as long as your want) ###############
# This is for reading the performance of your script
result['from_date'] = ''
result['to_date'] = ''
result['stoppedAt'] = str(pd.to_datetime(datetime.now()))
print(json.dumps(result))
print(json.dumps(db_tables))
except Exception as e:
#error out exeptions
result['status'] = '500'
result['records'] = 0
result['stoppedAt'] = str(pd.to_datetime(datetime.now()))
result['error'] = str(e)
print(json.dumps(result))