Christopher Sharkey

OpenSearch_CloudWatch_Alarms. CloudFormation stack automating the deployment of recomended cloudwatch alarms for OpenSearch

OpenSearch_Dashboard_Nginx_Proxy. Access the OpenSearch dashboard of a domin deployed in a private subnet via. a Nginx Proxy

Flink_Kinesis_Data_Analytics. Apache Flink examples designed to be run by AWS Kinesis Data Analytics (KDA).

Dev_Notes. Useful notes for setting up servers and explaining a few concepts

OpenSearch_kNN_Vector_Search. Tokenize and convert sample text data into vectors using BERT. Load the vector representation of the text to OpenSearch and use kNN for semantic search

OpenSearch_Index_Shard_Size. Example covering ideal shard size + how to adjust # of primary, replicate shards for an index

BM25_Search_Example. Example to help understand how the BM25 term based ranking model works in search applications

Iceberg_EMR_Athena. Resources from an virtual tech talk / workshop - Set Up and Use Apache Iceberg Tables on Your Data Lake

OpenSearch_Resource_Flow_Chart. Flow chart to help you navigate the OpenSearch resources I have built

Iceberg_update_metadata_script. Python script that will update S3 file paths in Iceberg metadata files (metadata.json + AVRO)

OpenSearch_Log_Analytics. Introduction workshop to log analytics on AWS OpenSearch

GitHub_Insigths_History. Script to collect GitHub repository views and unique visits

Flink_Late_Arriving_Date_Event_Order. Helps explain how Flink handles late arriving data and the effects on message order

EMR_Studio_Hudi. Apache Hudi examples designed to be run on AWS Elastic Map Reduce (EMR) via. EMR Studio or EMR Notebooks

OpenSearch_Refresh_Interval. Example covering how to adjust the refresh interval on an OpenSearch index

OpenSearch_Neural_Search. OpenSearch Neural Search example. Load BERT to OpenSearch and create embeddings as data is indexed. Use the embedding to preform vector search

OpenSearch_Kafka_Anomaly_Detection. Log data -> Kafka (MSK) - Lambda -> OpenSearch - Anomaly Detection

Outlook_MSG_Parser_Python. Python script to process emails saved with .msg file extension

Cosine_Similarity_Search_Example. Example to help understand how to use hugging face sentance_transformers to encode searchable text into embeddings AND how to use cosine similarity search to determine similarity between a search prompt and the embeddings

CloudFormation_Examples. Example YAML CloudFormation stacks

OpenSearch_API_Examples. Example API calls to set up OpenSearch (via. Python) for anomaly detection, cross cluster replication, load sample data ...

Iceberg_Glue_from_JARs. Configure any version of Apache Iceberg with AWS Glue by installing Iceberg from JAR files

EMR_Studio_Delta_Lake. Deltalake examples designed to be run on AWS Elastic Map Reduce (EMR) via. EMR Studio or EMR Notebooks

Glue_Aggregate_Small_Files. PySpark script to aggregate small parquet files in a prefix into larger files. Designed to be run on AWS Glue

Flink_Kinesis_Data_Analytics_Scala. Scala Apache Flink examples designed to be run by AWS Kinesis Data Analytics (KDA)

DataZone_Demo_FSI. Prebuilt demo of Amazon DataZone using fake data for topics in the financial services industry

OpenSearch_Audit_Logs. Breif explanation of REST layer audit logs for Amazon OpenSearch Service

Glue_Spark_History_Server. Host a Docker container for the Spark history server / Spark UI of AWS Glue jobs

Iceberg_Glue_register_table. Example using the Iceberg register_table command with AWS Glue and Glue Data Catalog

EMR_Studio_Iceberg. Apache Icebery examples designed to be run on AWS Elastic Map Reduce (EMR) via. EMR Studio or EMR Notebooks

OpenSearch_Sigv4_IAM_Auth. Authenticate with OpenSearch via. IAM Sigv4

EMR_Studio_Stock_Price_Demo. Demo EMR Studio notebook using PySpark to explore Stock Price Data

MongoDB_Streams_Glue_Iceberg. Process DynamoDB change streams via. AWS Glue w Iceberg to keep a copy of a collection in S3 upto date

Glue_Examples. PySpark code samples designed for AWS Glue

SecurityLake_AmazonSecurityLakeMetaStoreManager. CloudFormation to automate the deployment of the required IAM roles for AWS Security Lake

RESTful_API. A project creating a RESTful API using express.js and node.js

DS_220_Project. Web application utilizing MongoDB

Restaurant_Order_Web_Application. SWENG 894 Capstone Project

RESTful_API_w_node-restful. A project creating a RESTful API using the package node-restful

current_Date_Time. API to obtain the current Date and Time

EMR_Studio_Deployment. Example Jupyter notebook for EMR Studio

S3_Server_Access_Logs_ev2900-sharkech-public. Python script to download and parse S3 server access logs, to determine usage of CloudFormation stacks in other repositories

OpenSearch_DeletedDocuments. Examples explaining how deletes work in OpenSearch