$\def\ZZ{\mathbb{Z}}\def\QQ{\mathbb{Q}}\def\Spec{\mathrm{Spec}}\def\fp{\mathfrak{p}}\def\fq{\mathfrak{q}}\def\FF{\mathbb{F}}$This is one of two answers recording my partial progress. The point of this answer is to show that it is enough to resolve the question for the case where $A_1=A_2$ and both are the ring of integers of a number field $K$, with $K$ Galois over $\QQ$ and ramified only at $p$ and $\infty$ (for some prime $p$).
The other answer will contain my partial progress on that problem.

We begin with the following observation: Let $A_1 \subseteq B_1$ and $A_2 \subseteq B_2$ be free $\ZZ$-modules. Let $\zeta$ be an element of $A_1 \otimes A_2$ which factors as $b_1 \otimes b_2$ in $B_1 \otimes B_2$, and suppose that $b_1$ and $b_2$ are not divisible (in $B_1$ and $B_2$) by any integer $k \geq 2$. Then $b_j \in A_j$. Proof: Let $\pi_j$ be the projection $B_j \to B_j/A_j$. Since $\zeta \in A_1 \otimes A_2$, we have $(\pi_1 \otimes \mathrm{Id})(\zeta) =0$. So $\pi_1(b_1) \otimes b_2=0$ in $(B_1 / A_1) \otimes B_2$. Since $B_2$ is free over $\mathbb{Z}$ and $b_2$ is not divisible by any integer $\geq 2$, this means that $\pi_1(b_1)=0$, so $b_1 \in A_1$. Similarly, $b_2 \in A_2$. $\square$.

Also, notice that, in a ring finite over $\mathbb{Z}$, a root of unity can never have a nontrivial integer divisor, so when $B_1$ and $B_2$ are rings and $b_1$ and $b_2$ are roots of unity, that hypothesis is free. Thus, we deduce a **key trick**: If we can prove the theorem for a pair of rings $B_1$, $B_2$, we automatically deduce it for any subrings $A_1$, $A_2$.

Let $R_j$ be the radical of $A_j$ and let $D_j = A_j/R_j$. Since $A_j$ is connected, $D_j$ is a domain and $D_j \otimes \QQ$ is a field, say $K_j$. From the theory of Artin algebras over a field, we can split the exact sequence $0 \to (R_j \otimes \QQ) \to (A_j \otimes \QQ) \to K_j \to 0$ such that $K_j$ becomes a subring of $A_j \otimes \QQ$; write $\rho_j$ for the retraction $A_j \otimes \QQ \to R_j \otimes \QQ$. Let $\hat{R}_j$ be the $A_j$-submodule of $R_j \otimes \QQ$ generated by $\rho_j(A_j)$; note that this is still a finite $\ZZ$-module since $A_j$ and $\rho_j(A_j)$ are. So $A_j \subseteq \hat{R}_j \oplus D_j$ and it suffices to prove the result for $\hat{R}_j \oplus D_j$. **We have reduced to the case that the radical of $A_j$ breaks off as a direct summand. **

Let $A$ be a finite free commutative $\ZZ$-algebra whose radical splits as a direct summand, say $A = D \oplus R$. We claim that all roots of unity of $A$ are in $D$. Proof: Recall that $R$ is nilpotent. Set $R(k) = \QQ R^k \cap R$, where the intersection takes place in $\QQ \otimes R$. So $R(k)=0$ for $k$ sufficiently large. Let $d+r$ be a root of unity of $A$ with $d \in D$ and $r \in R$. We will show, by induction on $k$, that $r \in R(2^k)$; the base case $k=0$ is by hyothesis. Let $(d+r)^n=1$. Since the projection $A \to D$ is a map of rings, we have $d^n=1$ and $d$ is a root of unity and, in particular, is a unit. So, suppose that $r \in R(2^k)$. Then $1=(d+r)^n \equiv 1+n d^{-1} r \bmod r^2$. So $nr \in R(2^k)^2$ and $r \in R(2^{k+1})$. So $r \in \bigcap R(2^k) = \{ 0 \}$ and we have established the claim.

We now reduce to the case that $A_1$ and $A_2$ have no radical. Let $A_j = D_j \oplus R_j$. Then $A_1 \otimes A_2$ splits as $(D_1 \otimes D_2) \oplus (D_1 \otimes R_2 \oplus R_1 \otimes D_2 \oplus R_1 \otimes R_2)$, with $D_1 \otimes D_2$ semisimple and $D_1 \otimes R_2 \oplus R_1 \otimes D_2 \oplus R_1 \otimes R_2$ the radical. So the result of the previous paragraph shows that all roots of unity in $A_1 \otimes A_2$ are in $D_1 \otimes D_2$, so it is enough to prove the result for $D_1$ and $D_2$. **We have reduced to the case that $A_1$ and $A_2$ have no radical.**

So $A_1$ injects into a direct sum of domains, $A_1 \subseteq \bigoplus D_1^i$ and likewise $D_2 \subseteq \bigoplus D_2^k$. Our goal in the next several steps is to reduce to the case where $A_1$ and $A_2$ are each domains. So assume that we know the claim for each tensor product $D_1^i \otimes D_2^{k}$.
For each case where $\Spec D_1^i$ and $\Spec D_1^j$ meet in $\Spec A_1$, choose a pair of primes $\fp_1^{ij} \in \Spec D_1^i$ and $\fp_1^{ji} \in \Spec D_1^j$ where they meet.
I'll write an element of $\bigoplus D_1^i$ and $(a(1), a(2), \dots, a(r))$ with $a(i) \in D_1^i$, so we have $a(i) \bmod \fp_1^{ij} = a(j) \bmod \fp_1^{ji}$.
We may and do replace $A_1$ by the larger subring of $\bigoplus D_1^i$ given by these congruences, and do the same thing for $A_2$.

We can write an element of $A_1 \otimes A_2$ as a matrix $\left[ z(i,k) \right]$ with $z(i,k) \in D_1^i \otimes D_2^k$. Note that we have $z(i,k) \bmod \fp_1^{ij} \otimes D_2^k = z(j,k) \bmod \fp_1^{ji} \otimes D_2^k$ and likewise for $z(i,\ell)$ and $z(j,\ell)$.
This element will be a root of unity iff each $z(i,k)$ is a root of unity; by hypothesis, this occurs only if $z(i,k)$ is of the form $\alpha(i,k) \otimes \beta(k,i)$ for $\alpha(i,k)$ a root of unity in $D_1^i$ and $\beta(k,i)$ a root of unity in $D_2^k$. So $\alpha(i,k) \otimes \beta(k,i) \bmod \fp_1^{ij} \otimes D_2^k = \alpha(j,k) \otimes \beta(k,j) \bmod \fp_1^{ji} \otimes D_2^k$.
This forces $\beta(k,i) = \pm \beta(k,j)$. Likewise, $\alpha(i,k) = \pm \alpha(i,\ell)$. So there are roots of unity $\alpha(i) \in D_1^i$ and $\beta(k) \in D_2^k$ such that $z(i,k) = \pm \alpha(i) \otimes \beta(k)$. Moreover $\alpha(i) \bmod \fp_1^{ij} = \alpha(j) \bmod \fp_1^{ji}$, so $\alpha:= (\alpha(1), \ldots,\alpha(r))$ is a root of unity in $A_1$. Likewise, we construct a root of unity $\beta = (\beta(1), \ldots, \beta(s))$ in $A_2$. Dividing by $\alpha \otimes \beta$, we are reduced to considering roots of unity of the form $z(i,k) = \pm 1$. We want to show such a matrix factors as a tensor product, meaning that it is rank $1$. It is enough to show that each $2 \times 2$ matrix $\left[ \begin{smallmatrix} z(i,k) & z(i,\ell) \\ z(j,k) & z(j, \ell) \\ \end{smallmatrix} \right]$ has rank $1$. If $D_1^i/\fp_1^{ij}$ has odd characteristic, then we deduce that $z(i,k) = z(j,k)$ and $z(i,\ell) = z(j,\ell)$ and we are done. We are similarly done if $D_2^k/\fp_2^{k\ell}$ has odd characteristic. So we may assume instead these quotient fields have even characteristic.

At this point, an example may help. Let $R = \{ (a(1), a(2)) \subset \ZZ^{\oplus 2} : a(1) \equiv a(2) \bmod 2 \}$, and let's understand the square roots of $1$ in $R \otimes_{\ZZ} R$. So $R \otimes_{\ZZ} R$ can be identified with $2 \times 2$ matrices of integers $\left[ \begin{smallmatrix} z(1,1) & z(1,2) \\ z(2,1) & z(2,2) \\ \end{smallmatrix} \right]$, namely, those which are $\ZZ$-linear combinations of $\left[ \begin{smallmatrix} a(1) b(1) & a(1) b(2) \\ a(2) b(1) & a(2) b(2)\\ \end{smallmatrix} \right]$ with $a(1) \equiv a(2) \bmod 2$ and $b(1) \equiv b(2) \bmod 2$. We observe that every such matrix has $z(1,1) - z(1,2) - z(2,1) + z(2,2) \equiv 0 \bmod 4$. So $\left[ \begin{smallmatrix} 1 & 1 \\ 1 &-1 \\ \end{smallmatrix} \right]$ is not in $R \otimes_{\ZZ} R$. We want to generalize this computation to other rings.

As an abelian group, we have $D_1^i/\fp_1^{ij} \cong (\ZZ/2)^{\oplus r}$ for some $r$. So we can choose an integer basis $e_1$, $e_2$, ... , $e_s$ of $D_1^i$ such that $2 e_1$, $2 e_2$, ... $2 e_r$, $e_{r+1}$, ..., $e_s$ is an integer basis of $\fp_1^{ij}$; further more, we may take $e_1 = 1$. Let $\lambda_i : D_1^i \to \ZZ$ be projection onto the coefficient of $e_1$. So, for $a \in A_1$, we have $\lambda_i(a(i)) \equiv \lambda_j(a(j)) \bmod 2$. Likewise, we have maps $\mu_k : D_2^k \to \ZZ$ such that $\mu_k(b(k)) \equiv \mu_{\ell}(b(\ell)) \bmod 2$. Then, for any $a \in A_1$ and $b \in A_2$, we have $\lambda_i(a(i)) \mu_k(b(k)) - \lambda_i(a(i)) \mu_{\ell}(b(\ell)) - \lambda_j(a(j)) \mu_k(b(k)) + \lambda_j(a(j)) \mu_{\ell}(b(\ell)) = ( \lambda_i(a(i)) - \lambda_j(a(j)) ) ( \mu_k(b(k)) - \mu_{\ell}(b(\ell)) ) \equiv 0 \bmod 4$.

Thus, for any matrix $\left[ \begin{smallmatrix} z(1,1) & z(1,2) \\ z(2,1) & z(2,2) \\ \end{smallmatrix} \right]$ in $A_1 \otimes A_2$, we have $(\lambda_i \otimes \mu_k) z(i,k) - (\lambda_i \otimes \mu_{\ell}) z(i,\ell) - (\lambda_j \otimes \mu_k) z(j,k) + (\lambda_j \otimes \mu_{\ell}) z(j,\ell) \equiv 0 \bmod 4$.
If all the $z(i,k)$ are $\pm 1$, then we deduce that $z(i,k) - z(j,k) - z(i,\ell) + z(k,\ell) \equiv 0 \bmod 4$.
But by checking all $16$ matrices of $\pm 1$'s, this forces the matrix to be rank $1$. We have shown that, if the result holds for all $D_1^i \otimes D_2^k$, then it holds for $A_1 \otimes A_2$. **We have reduced to the case that $A_1$ and $A_2$ are domains.**

So, $A_1$ and $A_2$ are domains and, being finite $\ZZ$-algebras, they are orders in some number fields $K_1$ and $K_2$. Let $K$ be a Galois number field containing $K_1$ and $K_2$, and replace $A_1$ and $A_2$ by their rings of integers in $K$. **Thus, we have reduced to the case that $A_1=A_2=:A$ is the ring of integers in some Galois number field $K$. **

At this point, it is convenient to set up some notation. Let $G = \mathrm{Gal}(K/\QQ)$. For $\omega = \sum a_j \otimes b_j \in A \otimes A$ and $g_1$, $g_2 \in G$, put $\omega(g_1, g_2) = \sum g_1(a_j) g_2(b_j)$. It is well known that $\omega$ is determined by the $|G|^2$ values $\omega(g_1, g_2)$ and, since $\omega(g g_1, g g_2) = g \omega(g_1, g_2)$, it is in fact determined by the $|G|$ values $\omega(1,g_2)$. In this manner, $A \otimes A$ is identified with a subring of $A^{\oplus |G|}$.

Now, the set of roots of unity in $A \otimes A$ is a finite abelian group. If $\Gamma_1 \subseteq \Gamma_2$ are finite abelian groups, and their $p$-parts are equal for all $p$, then $\Gamma_1 = \Gamma_2$. So it is enough to show (for all $p$) that the $p^k$ roots of unity in $A \otimes A$ are of the form $\zeta_1 \otimes \zeta_2$ for $\zeta_1$ and $\zeta_2$ $p^k$-th roots of unity in $A$.
Fix such a $p$.

We will now reduce to the case that $p$ is the only finite prime of $\QQ$ ramified in $K$. Suppose that $q$ is some other ramified prime, let $\mathfrak{q}$ be a prime of $A$ above $q$ and let $H$ be the ramification group. For any $\omega \in A \otimes A$, we have $\omega(h g_1, g_2) \equiv \omega(g_1, g_2) \bmod \mathfrak{q}$.
Now, suppose that $\zeta$ is a $p^k$-root of unity in $A \otimes A$, so every $\zeta(g_1, g_2)$ is a $p^k$-root of unity.
The $p^k$ roots of unity stay distinct modulo $\mathfrak{q}$. So we deduce that $\zeta(h_1 g_1, h_2 g_2) = \zeta(g_1, g_2)$ for all $h_1$, $h_2 \in H$.
Since $\zeta$ is determined by the $\zeta(g_1, g_2)$, we deduce that $\zeta$ is invariant under $H \times H$. For a free $\ZZ$-module $A$ with an action of a finite group $H$, we have $(A \otimes A)^{H \times H} = (A^H) \otimes (A^H)$. So we may replace $K$ by the subfield $K^H$ and replace $A$ by the ring of integers in $K^H$. Repeating this argument for each of the primes above $q$, we shrink $K$ down to the minimal subextension unramified at $q$. Running the same argument for every finite prime $q \neq p$, **we have reduced to the case that $K$ is only ramified above $\{ p, \infty \}$.**