Using EFS for persistent storage

Percona Monitoring and Management (PMM) PMM 2.x
1

I am hosting PMM server on AWS ECS. In the process, I will be replacing ECS tasks. But ECS doesn’t support docker create command, hence I can’t create a data container.

So I am using EFS for persistent storage so that new tasks can access old data from EFS. But whenever I create new task and delete old one, it gives 500/502 errors.

Has anyone got PMM server to work with EFS?
What configuration do I need to take care of?

2

Hello @ashishps1 ,

I don’t think we have tested ECS + EFS configuration. ECS tasks have various use cases, but usually I see them as something ephemeral or used for batch-processing.

I cannot think of any particular reason for 500/502 errors, but it is NFS and it can be anything :slight_smile:

Could you please share your task definition? (don’t forget to strip sensitive data). We or someone in the community might try to reproduce it.
You can get it with aws ecs describe-task-definition --task-definition <TASK>.

1 Like
3

Here is the task definition I am using:

{
    "taskDefinition": {
        "taskDefinitionArn": "<arn>",
        "containerDefinitions": [
            {
                "name": "pmm",
                "image": "<ecr_image>",
                "cpu": 0,
                "memoryReservation": 1024,
                "portMappings": [
                    {
                        "containerPort": 443,
                        "hostPort": 443,
                        "protocol": "tcp"
                    }
                ],
                "essential": true,
                "environment": [],
                "mountPoints": [
                    {
                        "sourceVolume": "pmm-efs",
                        "containerPath": "/srv"
                    }
                ],
                "volumesFrom": [],
                "readonlyRootFilesystem": false,
                "logConfiguration": {
                    "logDriver": "awslogs",
                    "options": {
                        "awslogs-group": "/ecs/pmm/dev-productivity-20211206105022752900000001",
                        "awslogs-region": "us-west-2",
                        "awslogs-stream-prefix": "pmm-"
                    }
                }
            }
        ],
        "family": "pmm-dev-productivity-app",
        "taskRoleArn": "arn:aws:iam::<account_id>:role/pmm-task-role-20211206100923023800000001",
        "executionRoleArn": "arn:aws:iam::<account_id>:role/pmm-task-exec-role-20211122080554077800000001",
        "networkMode": "awsvpc",
        "revision": 6,
        "volumes": [
            {
                "name": "pmm-efs",
                "efsVolumeConfiguration": {
                    "fileSystemId": "fs-081ea8e6b2222eb2a",
                    "rootDirectory": "/",
                    "transitEncryption": "DISABLED",
                    "authorizationConfig": {
                        "iam": "DISABLED"
                    }
                }
            }
        ],
        "status": "ACTIVE",
        "requiresAttributes": [
            {
                "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
            },
            {
                "name": "ecs.capability.execution-role-awslogs"
            },
            {
                "name": "ecs.capability.efsAuth"
            },
            {
                "name": "com.amazonaws.ecs.capability.ecr-auth"
            },
            {
                "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
            },
            {
                "name": "ecs.capability.efs"
            },
            {
                "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
            },
            {
                "name": "com.amazonaws.ecs.capability.task-iam-role"
            },
            {
                "name": "com.amazonaws.ecs.capability.docker-remote-api.1.25"
            },
            {
                "name": "ecs.capability.execution-role-ecr-pull"
            },
            {
                "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
            },
            {
                "name": "ecs.capability.task-eni"
            }
        ],
        "placementConstraints": [
            {
                "type": "memberOf",
                "expression": "attribute:market == spot"
            }
        ],
        "compatibilities": [
            "EC2"
        ],
        "requiresCompatibilities": [
            "EC2"
        ],
        "registeredAt": 1638787832.575,
        "registeredBy": "arn:aws:iam::<account_id>:user/asingh-api"
    }
}

I have pulled the image from Docker Hub and saved in ECR. The version is 2.23.0.

1 Like