Drop rows based on conditions (longitudinal dataset)

You may try to group the data by ID, check the unique drug type for each patient and ensure it contains only CD and EF, filter the groups where this condition is met. Return the filtered dataframe.

import pandas as pd

# Sample Data
data = {
    'ID': ['A', 'A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C'],
    'drug': ['X', 'X', 'Y', 'Z', 'V', 'V', 'W', 'W', 'W', 'Z', 'V', 'V'],
    'drug sequence': [1, 1, 2, 3, 4, 4, 1, 1, 2, 1, 2, 2],
    'drug type': ['AB', 'AB', 'CD', 'CD', 'EF', 'EF', 'GH', 'GH', 'GH', 'CD', 'EF', 'EF']
}

df = pd.DataFrame(data)

# Filter the dataframe
def filter_patients(group):
    unique_drug_types = set(group['drug type'])
    return unique_drug_types == {'CD', 'EF'}

filtered_df = df.groupby('ID').filter(filter_patients)

# Display the result
print(filtered_df)

So, the result would be:

  ID drug  drug sequence drug type
9   C    Z              1        CD
10  C    V              2        EF
11  C    V              2        EF

You can use a set equality with groupby.transform and boolean indexing:

out = df[
    df.groupby('ID')['drug type'].transform(lambda x: set(x) == {'CD', 'EF'})
]

Output:

   ID drug  drug sequence drug type
9   C    Z              1        CD
10  C    V              2        EF
11  C    V              2        EF

Intermediate:

df.groupby('ID')['drug type'].transform(lambda x: set(x) == {'CD', 'EF'})

0     False
1     False
2     False
3     False
4     False
5     False
6     False
7     False
8     False
9      True
10     True
11     True
Name: drug type, dtype: bool

Something like that should work.

data = {'ID': ['A', 'A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C'],
        'drug': ['X', 'X', 'Y', 'Z', 'V', 'V', 'W', 'W', 'W', 'Z', 'V', 'V'],
        'drug sequence': [1, 1, 2, 3, 4, 4, 1, 1, 2, 1, 2, 2],
        'drug type': ['AB', 'AB', 'CD', 'CD', 'EF', 'EF', 'GH', 'GH', 'GH', 'CD', 'EF', 'EF']}

df = pd.DataFrame(data)

patient_drugs = df.groupby('ID')['drug type'].unique()
valid_patients = patient_drugs[patient_drugs.apply(lambda x: set(x) == {'CD', 'EF'})].index
filtered_df = df[df['ID'].isin(valid_patients)]

print(filtered_df)