SoftInputMode not working with a simple RelativeLayout

I've been working on an Android app and I have a simple Relativelayout containing multiple textiviews and a button as shown in a pseudocode example below.

The problem is that whenever I open any of the text fields I have the textfield on the bottom and the button both get covered by the keyboard.

I have SoftInputMode set to adjustPan in my AndroidManifest file and I have tried setting it to adjustResize but the elements still stay blocked.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://ift.tt/nIICcg"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:paddingLeft="16dp"
android:paddingRight="16dp"
android:paddingTop="25dp"
android:gravity="top|center"
>
<EditText
    android:id="@+id/input_1"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:hint="@string/1"
    android:paddingTop="25dp"
    android:inputType="number"
    />
<EditText
    android:id="@+id/input_2"
    android:layout_below="@id/input_1"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:hint="@string/1"
    android:paddingTop="25dp"
    android:inputType="number"
    />

<EditText
    android:id="@+id/input_3"
    android:layout_below="@id/input_2"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:hint="@string/1"
    android:paddingTop="25dp"
    android:inputType="number"
    />

<Button
    android:id="@+id/button"
    android:layout_below="@id/input_3"
    android:layout_height="wrap_content"
    android:layout_width="match_parent"
    android:layout_centerHorizontal="true"
    android:text="@string/button"
   />

Is there anything really simple I might be missing?