엉뚱한 녀석의 블로그 [quirky guy's Blog]

Automating AWS WAF IP Blocklists with Lambda (S3 + EventBridge, CloudFront WAFv2)

—

제공

black K
안 ,

Back when I led an infra/security team at a finance-related company, we needed to automatically apply the security provider’s periodic IP block list to AWS WAF. I’m documenting a slightly cleaned-up version of that setup here.

Goals

  • Ingest block IP lists (.txt, line-by-line) from the security provider and automatically apply them to a WAF IP set.
  • Immediate update when a file is uploaded to S3, plus a daily full aggregation (only files modified within the last 90 days) and re-apply.
  • Target a CloudFront-attached WAFv2 (Region us-east-1, Scope=CLOUDFRONT) for updates.

Why do we need two Lambda functions?

The source format varies by provider/environment:

  • Case A: Already in CIDR format (e.g., 203.0.113.0/24, 198.51.100.10/32)
    → You can push lines directly into the WAF IP set Addresses.
  • Case B: Non-CIDR single IPs (e.g., 203.0.113.5, 198.51.100.10)
    → WAF only accepts CIDRs, so convert to x.x.x.x/32 before applying.

If formats change over time or you have multiple providers, supporting both cases explicitly makes operations easier. (You could auto-detect in one function, but here we keep two clear variants for safer separation.)

End-to-end Flow

  • Management account web server: upload .txt →
  • Operations account S3 bucket (ipblocklists) stores the file →
  • S3 ObjectCreated triggers Lambda → Update WAF IP set
  • EventBridge (daily) → Lambda → fetch ip.php aggregated output (within last 90 days) → Update WAF IP set again
  • Lambda must reach ip.php, so place it in a VPC with NAT for outbound 443.
  • The web EC2 security group allows Inbound 443 (source: NAT Gateway).

Permissions / Policies — adapt to your environment

Management Account EC2 Role: ipblock-ec2

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:*", "s3-object-lambda:*"],
    "Resource": [
      "arn:aws:s3:::ipblocklists",
      "arn:aws:s3:::ipblocklists/*"
    ]
  }]
}

Operations Account S3 Bucket Policy (allow access from mgmt account EC2)

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::88853452443:role/ipblock-ec2" },
    "Action": "s3:*",
    "Resource": [
      "arn:aws:s3:::ipblocklists",
      "arn:aws:s3:::ipblocklists/*"
    ]
  }]
}

Operations Account Lambda Role (ipblock role)

  • AwsWafFullAccess
  • CreateNetworkInterface-Lambda (required when going out via VPC+NAT)
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ec2:DescribeNetworkInterfaces",
      "ec2:CreateNetworkInterface",
      "ec2:DeleteNetworkInterface",
      "ec2:DescribeInstances",
      "ec2:AttachNetworkInterface"
    ],
    "Resource": "*"
  }]
}

Note: In this design, Lambda doesn’t read S3 directly, so AmazonS3ReadOnlyAccess is not required.

Triggers & Network

  • S3: ObjectCreated:* → Lambda
  • EventBridge (daily): cron(12 00 ? * * *) (UTC 00:12) → Lambda
  • Lambda VPC SG: Inbound deny, Outbound TCP 443 allow
  • Web EC2 SG: Inbound 443 allow (Source: NAT Gateway)

Lambda Python Code (Case A: source is already CIDR)

ip.php emits x.x.x.x/nn lines. We push them directly to the WAF IP set. Some HTML <br /> tags may slip in—strip them out.

import logging
import boto3
import urllib.request

from ipaddress import ip_network

# 고정값 (CloudFront WAFv2)
WAF_REGION = 'us-east-1'
WAF_SCOPE  = 'CLOUDFRONT'

# 수동 지정
IPV4_SET_NAME = "test"  # WAF IP set 이름
IPV4_SET_ID   = "359fec83-b340-4c16-9bba-7b91f92eb7ee"  # WAF IP set ID

INFO_LOGGING = 'false'

def fetch_ip_list_from_url(url: str) -> list[str]:
    resp = urllib.request.urlopen(url, timeout=10)
    lines = resp.read().decode('utf-8', errors='ignore').splitlines()
    # 혹시 섞인 <br /> 제거
    return [line.replace('<br />', '').strip() for line in lines if line.strip()]

def lambda_handler(event, context):
    try:
        if len(logging.getLogger().handlers) > 0:
            logging.getLogger().setLevel(logging.ERROR)
        else:
            logging.basicConfig(level=logging.ERROR)
        if INFO_LOGGING == 'true':
            logging.getLogger().setLevel(logging.INFO)

        # CIDR 라인만 출력하는 ip.php
        ip_list = fetch_ip_list_from_url('https://example.com/ip.php')

        # 유효성(형식) 검증 & 정규화 (예: 1.2.3.4/32 → 1.2.3.4/32)
        cidrs: list[str] = []
        for raw in ip_list:
            try:
                cidrs.append(str(ip_network(raw, strict=False)))
            except Exception:
                logging.info(f"Skip invalid line (not CIDR): {raw}")

        update_waf_ipset(IPV4_SET_NAME, IPV4_SET_ID, cidrs)
        return {'status': f'IP 세트 업데이트 완료: {len(cidrs)} entries'}

    except Exception as e:
        logging.error('에러 발생', exc_info=True)
        return {'status': f'에러 발생: {e}'}

def update_waf_ipset(ipset_name: str, ipset_id: str, addresses: list[str]) -> None:
    waf = boto3.client('wafv2', region_name=WAF_REGION)
    lock_token = get_lock_token(waf, ipset_name, ipset_id)

    waf.update_ip_set(
        Name=ipset_name,
        Scope=WAF_SCOPE,   # CLOUDFRONT
        Id=ipset_id,
        Addresses=addresses,
        LockToken=lock_token
    )

def get_lock_token(waf, ipset_name: str, ipset_id: str) -> str:
    res = waf.get_ip_set(Name=ipset_name, Scope=WAF_SCOPE, Id=ipset_id)
    return res['LockToken']

Lambda Python Code (Case B: source is non-CIDR single IPs → convert to /32)

Using ip_network(x, strict=False) converts 1.2.3.4 to 1.2.3.4/32.

import logging
import boto3
import urllib.request
from ipaddress import ip_network

# 고정값 (CloudFront WAFv2)
WAF_REGION = 'us-east-1'
WAF_SCOPE  = 'CLOUDFRONT'

# 수동으로 지정되어야 하는 변수들
IPV4_SET_NAME = "test"  # AWS WAF IP 세트 이름
IPV4_SET_ID   = "359fec83-b340-4c16-9bba-7b91f92eb7ee"  # AWS WAF IP 세트 ID

INFO_LOGGING = 'false'

def fetch_ip_list_from_url(url: str) -> list[str]:
    resp = urllib.request.urlopen(url, timeout=10)
    lines = resp.read().decode('utf-8', errors='ignore').splitlines()
    # <br /> 제거 및 공백 제거
    return [line.replace('<br />', '').strip() for line in lines if line.strip()]

def generate_cidr_from_ip_list(ip_list: list[str]) -> list[str]:
    """비-CIDR 단일 IP 목록을 CIDR(/32)로 정규화.
       이미 CIDR이면 그대로 정규화(엄격성 완화). 잘못된 라인은 스킵."""
    cidrs: list[str] = []
    for raw in ip_list:
        try:
            # strict=False → 1.2.3.4 → 1.2.3.4/32 로 처리
            net = ip_network(raw, strict=False)
            # IPv4만 받으려면 아래에서 v4만 필터 가능 (필요 시):
            if net.version == 4:
                cidrs.append(str(net))
        except Exception:
            # 잘못된 라인/형식은 스킵
            logging.info(f"Skip invalid line: {raw}")
    # 중복 제거
    return sorted(set(cidrs))

def update_waf_ipset(ipset_name: str, ipset_id: str, addresses: list[str]) -> None:
    waf = boto3.client('wafv2', region_name=WAF_REGION)
    lock_token = get_lock_token(waf, ipset_name, ipset_id)
    waf.update_ip_set(
        Name=ipset_name,
        Scope=WAF_SCOPE,
        Id=ipset_id,
        Addresses=addresses,
        LockToken=lock_token
    )
    logging.info(f'Updated IPSet "{ipset_name}" with {len(addresses)} entries')

def get_lock_token(waf, ipset_name: str, ipset_id: str) -> str:
    res = waf.get_ip_set(Name=ipset_name, Scope=WAF_SCOPE, Id=ipset_id)
    return res['LockToken']

def lambda_handler(event, context):
    try:
        if len(logging.getLogger().handlers) > 0:
            logging.getLogger().setLevel(logging.ERROR)
        else:
            logging.basicConfig(level=logging.ERROR)
        if INFO_LOGGING == 'true':
            logging.getLogger().setLevel(logging.INFO)

        # 외부 URL에서 IP 주소 목록 가져오기
        ip_list = fetch_ip_list_from_url('https://example.com/ip.php')

        # 비-CIDR 단일 IP → /32 CIDR로 정규화
        cidr_ip_list = generate_cidr_from_ip_list(ip_list)

        # AWS WAF IP 세트 업데이트
        update_waf_ipset(IPV4_SET_NAME, IPV4_SET_ID, cidr_ip_list)
        return {'status': f'IP 세트 업데이트 완료: {len(cidr_ip_list)} entries'}

    except Exception as e:
        logging.error('에러 발생', exc_info=True)
        return {'status': f'에러 발생: {e}'}

Operations Tips

  • If you push very large lists, you may hit WAF IP set entry limits. Consider deduping/aggregation (merge into subnets) to reduce entries.
  • CloudFront WAF must be called in us-east-1 with Scope="CLOUDFRONT".

Management Account Web Server

index.php – main page

<?php
session_start();
if (!isset($_SESSION["username"])) { header("Location: login.php"); exit; }
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>WAF IP SET: IP Block 설정</title>
<style>
body{font-family:Arial,sans-serif;background:#f9f9f9;margin:0;padding:0}
.container{max-width:800px;margin:20px auto;padding:20px;background:#fff;border-radius:10px;box-shadow:0 0 10px rgba(0,0,0,.1)}
h1,h2{color:#333;margin-top:0}
.section{margin-bottom:30px}
.btn{background:#007bff;color:#fff;border:none;padding:10px 20px;border-radius:5px;cursor:pointer;transition:.3s}
.btn:hover{background:#0056b3}
ul{list-style:none;padding:0;margin:0}
ul li{margin-bottom:5px}
ul li a{text-decoration:none;color:#007bff}
ul li a:hover{text-decoration:underline}
.file-upload-form{display:flex;align-items:center;margin-bottom:15px}
.file-upload-form input[type="file"]{flex:1;margin-right:10px;padding:8px;border:1px solid #ccc;border-radius:5px}
</style>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script>$(function(){$("#showFilesBtn").click(function(){var w=window.open('get_files.php','_blank');w.focus();});});</script>
</head>
<body>
<div class="container">
  <h1>WAF IP SET: IP Block 설정</h1>
  <div class="section">
    <h2>**** 업로드 파일 설정 ****</h2>
    <form action="upload.php" method="post" enctype="multipart/form-data" class="file-upload-form">
      <input type="file" name="file" id="fileToUpload">
      <input type="submit" value="업로드 실행" class="btn">
    </form>
  </div>
  <div class="section">
    <h2>**** 업로드된 파일 리스트 보기 ****</h2>
    <button id="showFilesBtn" class="btn">업로드된 파일 리스트 보기</button>
  </div>
  <div class="section">
    <h2>**** IP 리스트 정렬 ****</h2>
    <ul><li><a href="ip.php">IP 차단 목록 보기</a></li></ul>
  </div>
</div>
</body>
</html>

ip.php – print only files with LastModified within 90 days

<?php

// AWS SDK 로드

require 'vendor/autoload.php';

use Aws\S3\S3Client;

use Aws\Exception\AwsException;

// S3 클라이언트 생성

$s3 = new S3Client([

'version' => 'latest',

'region' => 'ap-northeast-2', // AWS 리전 설정

]);

// S3 버킷 이름

$bucketName = 'ipblocklists';

try {

// S3 버킷에서 파일 목록 가져오기

$objects = $s3->listObjectsV2([

'Bucket' => $bucketName,

]);

// 현재 시간에서 90일 전의 타임스탬프 계산

$ninetyDaysAgo = strtotime("-90 days");

// 각 파일의 내용 출력

$contents = '';

foreach ($objects['Contents'] as $object) {

$fileName = $object['Key'];

$lastModified = strtotime($object['LastModified']);

// 파일의 수정 날짜가 90일 이내인 경우에만 출력

if ($lastModified > $ninetyDaysAgo) {

try {

// S3에서 파일 내용 가져오기

$result = $s3->getObject([

'Bucket' => $bucketName,

'Key' => $fileName,

]);

// 파일 내용을 한 줄에 하나씩 추가

$contents .= nl2br(htmlspecialchars($result['Body']->getContents())) . "\n";

} catch (AwsException $e) {

$contents .= "파일 내용을 가져오는 중 오류 발생: " . $e->getMessage() . "\n";

}

}

}

// 파일 내용 출력

echo $contents;

} catch (AwsException $e) {

echo "파일 목록을 가져오는 중 오류 발생: " . $e->getMessage();

}

?>

upload.php – upload to S3 with AWS SDK

<?php
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\Exception\AwsException;

$s3 = new S3Client(['version'=>'latest','region'=>'ap-northeast-2']);
$bucketName = 'ipblocklists';

if (!empty($_FILES['file']) && $_FILES['file']['error'] === UPLOAD_ERR_OK && isset($_FILES['file']['tmp_name'])) {
  $fileName = $_FILES['file']['name'];
  $filePath = $_FILES['file']['tmp_name'];
  try {
    $result = $s3->putObject([
      'Bucket'=>$bucketName,'Key'=>$fileName,'Body'=>fopen($filePath,'rb'),
      'ACL'=>'public-read'
    ]);
    echo "파일이 성공적으로 업로드되었습니다. 업로드된 파일 URL: " . $result['ObjectURL'];
  } catch (AwsException $e) { echo "파일 업로드 중 오류 발생: " . $e->getMessage(); }
} else { echo "파일 업로드 중 오류 발생: 파일이 존재하지 않습니다."; }

get_files.php – list uploaded files (with download/delete links)

<?php
session_start();
if (!isset($_SESSION["username"])) { header("Location: login.php"); exit; }

require 'vendor/autoload.php';
use Aws\S3\S3Client;

$bucketName = 'ipblocklists';
$s3 = new S3Client(['version'=>'latest','region'=>'ap-northeast-2']);

try {
  $objects = $s3->listObjectsV2(['Bucket'=>$bucketName]);
  echo "<h2>업로드된 파일 리스트</h2><ul>";
  foreach ($objects['Contents'] as $object) {
    $fileName = $object['Key'];
    echo "<li>{$fileName} - <a href='download.php?file={$fileName}'>Download</a> | <a href='delete.php?file={$fileName}'>Delete</a></li>";
  }
  echo "</ul>";
} catch (AwsException $e) { echo "파일 목록을 가져오는 중 오류 발생: " . $e->getMessage(); }

download.php

<?php
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\Exception\AwsException;

$bucketName = 'ipblocklists';
$s3 = new S3Client(['version'=>'latest','region'=>'ap-northeast-2']);

if (isset($_GET['file'])) {
  $fileName = $_GET['file'];
  try {
    $result = $s3->getObject(['Bucket'=>$bucketName,'Key'=>$fileName]);
    header('Content-Type: ' . $result['ContentType']);
    header('Content-Disposition: attachment; filename="' . basename($fileName) . '"');
    echo $result['Body'];
  } catch (AwsException $e) { echo "파일을 다운로드하는 중 오류 발생: " . $e->getMessage(); }
} else { echo "파일 이름이 제공되지 않았습니다."; }

delete.php

<?php
require 'vendor/autoload.php';
use Aws\S3\S3Client;

$bucketName = 'ipblocklists';

if (isset($_GET['file'])) {
  $fileName = $_GET['file'];
  $s3 = new S3Client(['version'=>'latest','region'=>'ap-northeast-2']);
  try {
    $s3->deleteObject(['Bucket'=>$bucketName,'Key'=>$fileName]);
    echo "파일이 성공적으로 삭제되었습니다: $fileName";
  } catch (\Throwable $e) { echo "파일 삭제 중 오류 발생: " . $e->getMessage(); }
} else { echo "삭제할 파일이 지정되지 않았습니다."; }

login.php – login form

<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>Login</title></head>
<body>
<h2>Login</h2>
<form action="loginuse.php" method="post">
  <label for="username">Username:</label>
  <input type="text" id="username" name="username" required><br><br>
  <label for="password">Password:</label>
  <input type="password" id="password" name="password" required><br><br>
  <input type="submit" value="Login" name="submit">
</form>
</body>
</html>

loginuse.php – handles login (ID/password are hardcoded)

<?php
$username = "test@example.com";
$password = "password";

if ($_SERVER["REQUEST_METHOD"] == "POST") {
  $input_username = $_POST["username"];
  $input_password = $_POST["password"];

  if ($input_username === $username && $input_password === $password) {
    session_start();
    $_SESSION["username"] = $username;
    header("Location: index.php"); exit;
  } else {
    echo "<p>Invalid username or password. Please try again.</p>";
  }
}

AWS Environment & Security Notes

This guide is intended as a reference so you can adapt it to your environment. You can use it as-is, but if your S3 ACL/WAFv2 location/network differs, adjust accordingly.

  • loginuse.php hard-codes an ID/password. In production, move secrets out of code (env vars/parameter store/SSO), force HTTPS, and harden sessions.

Checklist

  • S3 ObjectCreated → Lambda
  • EventBridge (UTC 00:12) → Lambda (90-day full aggregation) → Korea time 09:12
  • Lambda config: Region us-east-1, Scope CLOUDFRONT
  • Lambda VPC Outbound 443 allowed, Web EC2 Inbound 443 allowed (source NAT GW)
  • ip.php: print only last-90-days files, one IP/CIDR per line
  • Choose Lambda by input format: Case A (already CIDR) or Case B (non-CIDR → /32)
ⓒ 2025 엉뚱한 녀석의 블로그 [quirky guy's Blog]. 본문 및 이미지를 무단 복제·배포할 수 없습니다. 공유 시 반드시 원문 링크를 명시해 주세요.
ⓒ 2025 엉뚱한 녀석의 블로그 [quirky guy's Blog]. All rights reserved. Unauthorized copying or redistribution of the text and images is prohibited. When sharing, please include the original source link.

코멘트

답글 남기기