Even the best security measures can’t prevent all security incidents. With the right tools, you can detect and respond to incidents quickly. The problem is not anymore that you have no tools at hand but that you have too many. How do you choose the right one?
Retrieval-Augmented Generation (RAG) solutions are an impressive way to talk to one’s data. One of the challenges of RAG solutions is the associated cost, often driven by the vector database. In a previous blog article I presented how to tackle this issue by using Athena with Locality Sensitive Hashing (LSH) as a knowledge database. One the of the main limitations with Athena is the latency and the low number of concurrent queries. In this new blog article, I present a new low-cost serverless solution that makes use of an embedded vector database, SQLite, to achieve a low cost while maintaining high concurrency.
This guide is for organizations currently using Solr and considering a migration to Amazon OpenSearch Service. We will explore the benefits of migrating, the process, and the key challenges, offering insights from real-world migrations to help you make an informed decision.
Business insights are frequently generated from aggregated data, like daily sales per market segment over time. In this blog post we’ll use Apache Airflow to build a data aggregation pipeline that utilizes Amazon Athena for the heavy lifting. We’ll cover best practices that you should follow to build a production-ready system.
The TPC-H dataset is commonly used to benchmark data warehouses or, more generally, decision support systems. It describes a typical e-commerce workload and includes benchmark queries to enable performance comparison between different data warehouses. I think the dataset is also useful to teach building different kinds of ETL or analytics workflows, so I decided to explore ways of making it available in Amazon Athena.
If you’re trying to use Apache Airflow to copy large objects in S3, you might have encountered issues where S3 complains about you sending an InvalidRequest. We will fix that in this post by writing a custom operator to handle the underlying problem.
You want to ship the very same code you have tested from test stage to production stage. But most of the time a CICD pipeline re-creates the code to deploy to production. So you do not deploy the tested code! With the AWS Lambda transport tool you can move the very same code you have tested to production or between stages in different accounts.
Even though I had opted out of performance measurement cookies, I noticed a lot of web requests that look like performance measurement in the AWS console. In this article I investigate what’s being sent and what we can do about it.
