Xenforo User Email field extraction using API

[jcase]

Hi,
We are trying to get the User data for our needs and when looked into the documentation we observed that Email is a locked field and can't be extracted easily.
We tried all the methods using User API and Super API but we couldn't get the Email field and shows us an error.
When used Super API, I got the below mentioned issue:

The remote server returned an error 403 forbidden
Status Code: Forbidden
Response body: code: do not have permission
message: You do not have permission to view this page or perform this action.

Any help would be appreciated. Can someone guide us?
Thanks!

 

[Sim]

The user needs Administrator privileges with the "Manage users and moderators" permission to read email addresses.

Basically, if a user can't see the information in the normal Admin UI, then they won't see it via the API.

 

[smozgur]

With super user key, pass "api_bypass_permissions=1" parameter.

/users/123?api_bypass_permissions=1

Note: It won't work for /users/ to list all users since the verbose output only works for individual GET requests.

 

[jcase]

Hi Smozgur,

The issue over here, we are trying to get all the users whoever came to our page along with their Email ids.
The one which I am using right now is User Key and I am successful in getting their usernames and when they exactly joined but Email field shows lock and Email is the main field for our analysis to see their purchases.

https://ourcompany/api/users/
and I am getting all the other fields like below except Email and Email shows it is locked field when looked into the documentation.


I tried using Super API key but it says I don't have permission with the same URL with endpoints.

You mentioned me that it won't work for users but I get all the other user information like username why couldn't I get the Email for that particular user and I see it is mentioned in the API documentation.

REST API | Developer Documentation | XenForo

Developer documentation for XenForo

xenforo.com

REST API endpoints (2.2)

This document assumes you have already created an API key and setup the necessary headers for your request. Learn more about using the API.

xenforo.com

Please suggest us and let me know if you have any questions.

Thanks,
Lakshmi

 

[smozgur]

As I mentioned, it is not going to work when you fetch all users, but only by the user ID individually.

The reason is that the API endpoint /users/ doesn't pass the verbose mode to the toApiResult() method, so it uses the default verbose mode, which is VERBOSITY_NORMAL.

However, the API endpoint /users/{id}/ passes the verbose mode as VERBOSITY_VERBOSE, which is necessary to expose the information you need.

There is of course a workaround, but it requires some custom programming.

Still, if you'd like to go that route: You need to extend \XF\Api\Controller\Users controller, and overwrite actionGet() method to inject Entity::VERBOSITY_VERBOSE, or basically 2 as integer, into the $users->toApiResults() method.

Then you can either user a super user key with api_bypass_permissions parameter as I explained previously, or you can use a user key belongs to a user who has "view profile" permissions.

 

[smozgur]

The issue over here, we are trying to get all the users whoever came to our page along with their Email ids.

Aha, I don't think you need /users/ in this case. You can use /users/{id}/ to fetch the information you need. If you use the actual visitor's account to connect to the API, then it will already allow to get their own information as $includePrivateProfile value will be true in the Entity::setupApiResultData method.

However, I don't know you project, its context, scope, and how it is supposed to work.

 

[jcase]

Every time when we need an Email for new users, it is not possible to enter each userid and get their email address.

I tried using User API key and mentioned user/{id}/ but when trying to get the columns, email field is showing null.
When looked into the documentation, it shows it is returned only if the permission are met - What kind of permissions do we need to meet?



I am not sure if I understood the below lines which you mentioned: Is there a documentation that I can get clear understanding on this?
There is of course a workaround, but it requires some custom programming.

Still, if you'd like to go that route: You need to extend \XF\Api\Controller\Users controller, and overwrite actionGet() method to inject Entity::VERBOSITY_VERBOSE, or basically 2 as integer, into the $users->toApiResults() method.

Then you can either user a super user key with api_bypass_permissions parameter as I explained previously, or you can use a user key belongs to a user who has "view profile" permissions.

 

[jcase]

One more small doubt: What do you mean by Verbosity_verbose over here? It would be much helpful if we have a separate documentation for this.
However, the API endpoint /users/{id}/ passes the verbose mode as VERBOSITY_VERBOSE, which is necessary to expose the information you need.

 

[Mike]

It might be best to expand on what you're trying to accomplish, because if you're just trying to extract the email addresses of your user base, you can do this via the control panel in the "email users" section by choosing the "Only generate a list of email addresses" option. No API access needed.

But as has otherwise been mentioned, the email address is only returned in the users/{id}/ endpoint. You could query users/ and then read the individual user IDs there to query the specific endpoint. If you're getting a no permission error, then the user you're accessing the API with doesn't have permissions to query that information. (If you are using a super user API key, you need to pass the user ID you are accessing as or it'll default to a guest user; alternatively, you can use the bypass permissions option: https://xenforo.com/xf2-docs/dev/rest-api/#accessing-the-api)

 

[jcase]

Hi Mike,
The main point over here is:
We are trying to understand who is using the system and obtain their basic activities from our system.

1. We are trying to get the user information into our database using this API. We got all the information like username, user_id, registered date, last activity but not Email field. Email is the main field which helps us to compare and see what are the customer activities from our side.

2. We just don't want only Emails, we need at least userid or username along with the Email so that we can compare and get the registered date and last activity of a individual user into our database on everyday basis.

3. You mentioned above like:
If you're just trying to extract the email addresses of your user base, you can do this via the control panel in the "email users" section by choosing the "Only generate a list of email addresses" option. No API access needed.
Can we extract Email address and username at least together into SQL database without using API?

4. I used the User API key and no error occurred and tried to get the Email address of one particular user id and the Email field shows blank values.
May be I didn't use Super User Key as I just used User API key.
I tried using Super API key also and when I used this, I couldn't even see the Email field in the preview data.

5. If I use the super API key, where should I pass the user id?

Thanks!

 

[jcase]

Hi,
Just wanted to know if there is any update on our issue.
Thanks!

 

[Mike]

Can we extract Email address and username at least together into SQL database without using API?

Yes, these are in the xf_user table.

4. I used the User API key and no error occurred and tried to get the Email address of one particular user id and the Email field shows blank values.
May be I didn't use Super User Key as I just used User API key.
I tried using Super API key also and when I used this, I couldn't even see the Email field in the preview data.

The email field either won't be present or it will have the value for the email in the database. There isn't any reason it should be blank.

If you don't see the email field at all, assuming you're calling users/{id}/, that would be a permission issue with the user the API request is being made as.

5. If I use the super API key, where should I pass the user id?

See the part about the XF-Api-User header in the accessing the API section of the docs. Alternatively, if using a super user key, there is an option to bypass permission checks entirely, which is explained there as well.

 

[adeel786]

@Mike

If you don't see the email field at all, assuming you're calling users/{id}/, that would be a permission issue with the user the API request is being made as.

How can I set this permission?

I am using guest api key, with only user:read scope

before the latest release, by using guest api key I can fetch the email.
Now all data is displaying except email