This WordPress filter hooks into WP_Http::_dispatch_request() to write all request and response details into a logfile.
It logs all HTTP requests (GET, POST, HEAD) that are sent and received by the WP_Http class, e.g. explicitly via $wp_http_object->request() or via wp_remote_request() or wp_remote_get() or wp_remote_post() or wp_remote_head().
The hook is useful for logging while debugging WordPress plugins that connect to other servers.
<?php
function wp_log_http_requests( $response, $args, $url ) {
// set your log file location here
$logfile = plugin_dir_path( __FILE__ ) . '/http_requests.log';
// parse request and response body to a hash for human readable log output
$log_response = $response;
if ( isset( $args['body'] ) ) {
parse_str( $args['body'], $args['body_parsed'] );
}
if ( isset( $log_response['body'] ) ) {
parse_str( $log_response['body'], $log_response['body_parsed'] );
}
// write into logfile
file_put_contents( $logfile, sprintf( "### %s, URL: %s\nREQUEST: %sRESPONSE: %s\n", date( 'c' ), $url, print_r( $args, true ), print_r( $log_response, true ) ), FILE_APPEND );
return $response;
}
// hook into WP_Http::_dispatch_request()
add_filter( 'http_response', 'wp_log_http_requests', 10, 3 );
?>
See also:
WordPress Plugin API Documentation: Filters
WordPress Function Reference: add_filter
Having the error message “Malformed UTF-8 character (fatal)” in my log files, I tried to handle this properly without letting the process die nor throwing away the whole string.
Having some research on Google I came up with following solution:
sub encode_utf_8 {
my $string = @_;
my $utf8_encoded = '';
eval {
$utf8_encoded = Encode::encode('UTF-8', $string, Encode::FB_CROAK);
};
if ($@) {
# sanitize malformed UTF-8
$utf8_encoded = '';
my @chars = split(//, $string);
foreach my $char (@chars) {
my $utf_8_char = eval { Encode::encode('UTF-8', $char, Encode::FB_CROAK) }
or next;
$utf8_encoded .= $utf_8_char;
}
}
return $utf8_encoded;
}
See also:
http://perldoc.perl.org/Encode.html#Handling-Malformed-Data
http://www.perlmonks.org/?node_id=839519
This is how you can dynamically set a custom HTTP User-Agent for your Perl requests to fake a device or browser for testing purpose or getting a device-specific version of a website.
WWW::Mechanize supports setting a custom user-agent with the constructor and after this gives a choice of 6 pre-defined basic user-agents ( $mech->agent_alias() ), only.
The following code demonstrates how to dynamically change the user-agent on a Mechanize object.
use WWW::Mechanize;
my $initial_user_agent = 'Mozilla/5.0 (Linux; U; Android 2.2; de-de; HTC Desire HD 1.18.161.2 Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1';
my @user_agents = (
'Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13',
'Mozilla/5.0 (iPad; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7D11',
'Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5',
);
# Set an initial custom header with the contructor
my $mech = WWW::Mechanize->new( agent => $initial_user_agent );
# get a page and print current URI (WWW::Mechanize follows redirections)
$mech->get( 'http://www.facebook.com' );
print sprintf( "User-Agent %s\n redirects to: %s\n\n", $initial_user_agent, $mech->uri() );
foreach my $http_user_agent (@user_agents) {
# dynamically set custom HTTP User-agents
$mech->add_header( 'User-agent' => $http_user_agent);
$mech->get( 'http://www.facebook.com' );
print sprintf( "User-Agent %s\n redirects to: %s\n\n", $http_user_agent, $mech->uri() );
}
# $ perl ./mechanize-user-agent.pl
# User-Agent Mozilla/5.0 (Linux; U; Android 2.2; de-de; HTC Desire HD 1.18.161.2 Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
# redirects to: http://m.facebook.com/?w2m&refsrc=http%3A%2F%2Fwww.facebook.com%2F&_rdr
#
# User-Agent Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13
# redirects to: http://www.facebook.com
#
# User-Agent Mozilla/5.0 (iPad; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7D11
# redirects to: http://m.facebook.com/?w2m&refsrc=http%3A%2F%2Fwww.facebook.com%2F&_rdr
#
# User-Agent Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
# redirects to: http://m.facebook.com/?w2m&refsrc=http%3A%2F%2Fwww.facebook.com%2F&_rdr