The source file subdomains_ex2.C with comments:

Subdomains Example 2 - Subdomain-Restricted Variables



This example builds on the fourth example program by showing how to restrict solution fields to a subdomain (or union of subdomains).



C++ include files that we need
        #include <iostream>
        #include <algorithm>
        #include <math.h>
        
Basic include file needed for the mesh functionality.
        #include "libmesh/libmesh.h"
        #include "libmesh/mesh.h"
        #include "libmesh/mesh_generation.h"
        #include "libmesh/exodusII_io.h"
        #include "libmesh/gnuplot_io.h"
        #include "libmesh/linear_implicit_system.h"
        #include "libmesh/equation_systems.h"
        
Define the Finite Element object.
        #include "libmesh/fe.h"
        
Define Gauss quadrature rules.
        #include "libmesh/quadrature_gauss.h"
        
Define the DofMap, which handles degree of freedom indexing.
        #include "libmesh/dof_map.h"
        
Define useful datatypes for finite element matrix and vector components.
        #include "libmesh/sparse_matrix.h"
        #include "libmesh/numeric_vector.h"
        #include "libmesh/dense_matrix.h"
        #include "libmesh/dense_vector.h"
        
Define the PerfLog, a performance logging utility. It is useful for timing events in a code and giving you an idea where bottlenecks lie.
        #include "libmesh/perf_log.h"
        
The definition of a geometric element
        #include "libmesh/elem.h"
        
        #include "libmesh/string_to_enum.h"
        #include "libmesh/getpot.h"
        
Bring in everything from the libMesh namespace
        using namespace libMesh;
        
        
        
Function prototype. This is the function that will assemble the linear system for our Poisson problem. Note that the function will take the \p EquationSystems object and the name of the system we are assembling as input. From the \p EquationSystems object we have acess to the \p Mesh and other objects we might need.
        void assemble_poisson(EquationSystems& es,
                              const std::string& system_name);
        
Exact solution function prototype.
        Real exact_solution (const Real x,
                             const Real y = 0.,
                             const Real z = 0.);
        
Begin the main program.
        int main (int argc, char** argv)
        {
Initialize libMesh and any dependent libaries, like in example 2.
          LibMeshInit init (argc, argv);
        
Declare a performance log for the main program PerfLog perf_main("Main Program");

Create a GetPot object to parse the command line
          GetPot command_line (argc, argv);
        
Check for proper calling arguments.
          if (argc < 3)
            {
              if (libMesh::processor_id() == 0)
                std::cerr << "Usage:\n"
                          <<"\t " << argv[0] << " -d 2(3)" << " -n 15"
                          << std::endl;
        
This handy function will print the file name, line number, and then abort. Currrently the library does not use C++ exception handling.
              libmesh_error();
            }
        
Brief message to the user regarding the program name and command line arguments.
          else
            {
              std::cout << "Running " << argv[0];
        
              for (int i=1; i<argc; i++)
                std::cout << " " << argv[i];
        
              std::cout << std::endl << std::endl;
            }
        
        
Read problem dimension from command line. Use int instead of unsigned since the GetPot overload is ambiguous otherwise.
          int dim = 2;
          if ( command_line.search(1, "-d") )
            dim = command_line.next(dim);
        
Skip higher-dimensional examples on a lower-dimensional libMesh build
          libmesh_example_assert(dim <= LIBMESH_DIM, "2D/3D support");
        
Create a mesh with user-defined dimension on the default MPI communicator.
          Mesh mesh (init.comm(), dim);
        
Read number of elements from command line
          int ps = 15;
          if ( command_line.search(1, "-n") )
            ps = command_line.next(ps);
        
Read FE order from command line
          std::string order = "SECOND";
          if ( command_line.search(2, "-Order", "-o") )
            order = command_line.next(order);
        
Read FE Family from command line
          std::string family = "LAGRANGE";
          if ( command_line.search(2, "-FEFamily", "-f") )
            family = command_line.next(family);
        
Cannot use discontinuous basis.
          if ((family == "MONOMIAL") || (family == "XYZ"))
            {
              if (libMesh::processor_id() == 0)
                std::cerr << "ex28 currently requires a C^0 (or higher) FE basis." << std::endl;
              libmesh_error();
            }
        
Use the MeshTools::Generation mesh generator to create a uniform grid on the square [-1,1]^D. We instruct the mesh generator to build a mesh of 8x8 \p Quad9 elements in 2D, or \p Hex27 elements in 3D. Building these higher-order elements allows us to use higher-order approximation, as in example 3.

          Real halfwidth = dim > 1 ? 1. : 0.;
          Real halfheight = dim > 2 ? 1. : 0.;
        
          if ((family == "LAGRANGE") && (order == "FIRST"))
            {
No reason to use high-order geometric elements if we are solving with low-order finite elements.
              MeshTools::Generation::build_cube (mesh,
                                                 ps,
        					 (dim>1) ? ps : 0,
        					 (dim>2) ? ps : 0,
                                                 -1., 1.,
                                                 -halfwidth, halfwidth,
                                                 -halfheight, halfheight,
                                                 (dim==1)    ? EDGE2 :
                                                 ((dim == 2) ? QUAD4 : HEX8));
            }
        
          else
            {
              MeshTools::Generation::build_cube (mesh,
        					 ps,
        					 (dim>1) ? ps : 0,
        					 (dim>2) ? ps : 0,
                                                 -1., 1.,
                                                 -halfwidth, halfwidth,
                                                 -halfheight, halfheight,
                                                 (dim==1)    ? EDGE3 :
                                                 ((dim == 2) ? QUAD9 : HEX27));
            }
        
          {
            MeshBase::element_iterator       el     = mesh.elements_begin();
            const MeshBase::element_iterator end_el = mesh.elements_end();
        
            for ( ; el != end_el; ++el)
              {
        	Elem* elem = *el;
        	const Point cent = elem->centroid();
                if (dim > 1)
                  {
        	    if ((cent(0) > 0) == (cent(1) > 0))
        	      elem->subdomain_id() = 1;
                  }
                else
                  {
        	    if (cent(0) > 0)
        	      elem->subdomain_id() = 1;
                  }
              }
          }
        
Print information about the mesh to the screen.
          mesh.print_info();
        
Create an equation systems object.
          EquationSystems equation_systems (mesh);
        
Declare the system and its variables. Create a system named "Poisson"
          LinearImplicitSystem& system =
            equation_systems.add_system<LinearImplicitSystem> ("Poisson");
        
        
          std::set<subdomain_id_type> active_subdomains;
        
        
Add the variable "u" to "Poisson". "u" will be approximated using second-order approximation.
          active_subdomains.clear(); active_subdomains.insert(0);
          system.add_variable("u",
                              Utility::string_to_enum<Order>   (order),
                              Utility::string_to_enum<FEFamily>(family),
        		      &active_subdomains);
        
Add the variable "v" to "Poisson". "v" will be approximated using second-order approximation.
          active_subdomains.clear(); active_subdomains.insert(1);
          system.add_variable("v",
                              Utility::string_to_enum<Order>   (order),
                              Utility::string_to_enum<FEFamily>(family),
        		      &active_subdomains);
        
Give the system a pointer to the matrix assembly function.
          system.attach_assemble_function (assemble_poisson);
        
Initialize the data structures for the equation system.
          equation_systems.init();
        
Print information about the system to the screen.
          equation_systems.print_info();
          mesh.print_info();
        
Solve the system "Poisson", just like example 2.
          equation_systems.get_system("Poisson").solve();
        
After solving the system write the solution to a GMV-formatted plot file.
          if(dim == 1)
          {
            GnuPlotIO plot(mesh,"Subdomains Example 2, 1D",GnuPlotIO::GRID_ON);
            plot.write_equation_systems("gnuplot_script",equation_systems);
          }
          else
          {
        #ifdef LIBMESH_HAVE_EXODUS_API
            ExodusII_IO (mesh).write_equation_systems ((dim == 3) ?
              "out_3.e" : "out_2.e",equation_systems);
        #endif // #ifdef LIBMESH_HAVE_EXODUS_API
          }
        
All done.
          return 0;
        }
        
        
        
        






We now define the matrix assembly function for the Poisson system. We need to first compute element matrices and right-hand sides, and then take into account the boundary conditions, which will be handled via a penalty method.
        void assemble_poisson(EquationSystems& es,
                              const std::string& system_name)
        {
It is a good idea to make sure we are assembling the proper system.
          libmesh_assert_equal_to (system_name, "Poisson");
        
Declare a performance log. Give it a descriptive string to identify what part of the code we are logging, since there may be many PerfLogs in an application.
          PerfLog perf_log ("Matrix Assembly");
        
Get a constant reference to the mesh object.
          const MeshBase& mesh = es.get_mesh();
        
The dimension that we are running
          const unsigned int dim = mesh.mesh_dimension();
        
Get a reference to the LinearImplicitSystem we are solving
          LinearImplicitSystem& system = es.get_system<LinearImplicitSystem>("Poisson");
        
A reference to the \p DofMap object for this system. The \p DofMap object handles the index translation from node and element numbers to degree of freedom numbers. We will talk more about the \p DofMap in future examples.
          const DofMap& dof_map = system.get_dof_map();
        
Get a constant reference to the Finite Element type for the first (and only) variable in the system.
          FEType fe_type = dof_map.variable_type(0);
        
Build a Finite Element object of the specified type. Since the \p FEBase::build() member dynamically creates memory we will store the object as an \p AutoPtr. This can be thought of as a pointer that will clean up after itself.
          AutoPtr<FEBase> fe (FEBase::build(dim, fe_type));
        
A 5th order Gauss quadrature rule for numerical integration.
          QGauss qrule (dim, FIFTH);
        
Tell the finite element object to use our quadrature rule.
          fe->attach_quadrature_rule (&qrule);
        
Declare a special finite element object for boundary integration.
          AutoPtr<FEBase> fe_face (FEBase::build(dim, fe_type));
        
Boundary integration requires one quadraure rule, with dimensionality one less than the dimensionality of the element.
          QGauss qface(dim-1, FIFTH);
        
Tell the finte element object to use our quadrature rule.
          fe_face->attach_quadrature_rule (&qface);
        
Here we define some references to cell-specific data that will be used to assemble the linear system. We begin with the element Jacobian * quadrature weight at each integration point.
          const std::vector<Real>& JxW = fe->get_JxW();
        
The physical XY locations of the quadrature points on the element. These might be useful for evaluating spatially varying material properties at the quadrature points.
          const std::vector<Point>& q_point = fe->get_xyz();
        
The element shape functions evaluated at the quadrature points.
          const std::vector<std::vector<Real> >& phi = fe->get_phi();
        
The element shape function gradients evaluated at the quadrature points.
          const std::vector<std::vector<RealGradient> >& dphi = fe->get_dphi();
        
Define data structures to contain the element matrix and right-hand-side vector contribution. Following basic finite element terminology we will denote these "Ke" and "Fe". More detail is in example 3.
          DenseMatrix<Number> Ke;
          DenseVector<Number> Fe;
        
This vector will hold the degree of freedom indices for the element. These define where in the global system the element degrees of freedom get mapped.
          std::vector<dof_id_type> dof_indices, dof_indices2;
        
Now we will loop over all the elements in the mesh. We will compute the element matrix and right-hand-side contribution. See example 3 for a discussion of the element iterators. Here we use the \p const_local_elem_iterator to indicate we only want to loop over elements that are assigned to the local processor. This allows each processor to compute its components of the global matrix.

"PARALLEL CHANGE"
          MeshBase::const_element_iterator       el     = mesh.local_elements_begin();
          const MeshBase::const_element_iterator end_el = mesh.local_elements_end();
        
          for ( ; el != end_el; ++el)
            {
Start logging the shape function initialization. This is done through a simple function call with the name of the event to log.
              perf_log.push("elem init");
        
Store a pointer to the element we are currently working on. This allows for nicer syntax later.
              const Elem* elem = *el;
        
Get the degree of freedom indices for the current element. These define where in the global matrix and right-hand-side this element will contribute to.
              dof_map.dof_indices (elem, dof_indices,0);
              dof_map.dof_indices (elem, dof_indices2,1);
        
std::cout << "dof_indices.size()=" << dof_indices.size() << ", dof_indices2.size()=" << dof_indices2.size() << std::endl;

Compute the element-specific data for the current element. This involves computing the location of the quadrature points (q_point) and the shape functions (phi, dphi) for the current element.
              fe->reinit (elem);
        
Zero the element matrix and right-hand side before summing them. We use the resize member here because the number of degrees of freedom might have changed from the last element. Note that this will be the case if the element type is different (i.e. the last element was a triangle, now we are on a quadrilateral).
              Ke.resize (std::max(dof_indices.size(), dof_indices2.size()),
        		 std::max(dof_indices.size(), dof_indices2.size()));
        
              Fe.resize (std::max(dof_indices.size(), dof_indices2.size()));
        
Stop logging the shape function initialization. If you forget to stop logging an event the PerfLog object will probably catch the error and abort.
              perf_log.pop("elem init");
        
Now we will build the element matrix. This involves a double loop to integrate the test funcions (i) against the trial functions (j).

We have split the numeric integration into two loops so that we can log the matrix and right-hand-side computation seperately.

Now start logging the element matrix computation
              perf_log.push ("Ke");
        
              for (unsigned int qp=0; qp<qrule.n_points(); qp++)
                for (unsigned int i=0; i<phi.size(); i++)
                  for (unsigned int j=0; j<phi.size(); j++)
                    Ke(i,j) += JxW[qp]*(dphi[i][qp]*dphi[j][qp]);
        
        
Stop logging the matrix computation
              perf_log.pop ("Ke");
        
Now we build the element right-hand-side contribution. This involves a single loop in which we integrate the "forcing function" in the PDE against the test functions.

Start logging the right-hand-side computation
              perf_log.push ("Fe");
        
              for (unsigned int qp=0; qp<qrule.n_points(); qp++)
                {
fxy is the forcing function for the Poisson equation. In this case we set fxy to be a finite difference Laplacian approximation to the (known) exact solution.

We will use the second-order accurate FD Laplacian approximation, which in 2D on a structured grid is

u_xx + u_yy = (u(i-1,j) + u(i+1,j) + u(i,j-1) + u(i,j+1) + -4*u(i,j))/h^2

Since the value of the forcing function depends only on the location of the quadrature point (q_point[qp]) we will compute it here, outside of the i-loop
                  const Real x = q_point[qp](0);
        #if LIBMESH_DIM > 1
                  const Real y = q_point[qp](1);
        #else
                  const Real y = 0;
        #endif
        #if LIBMESH_DIM > 2
                  const Real z = q_point[qp](2);
        #else
                  const Real z = 0;
        #endif
                  const Real eps = 1.e-3;
        
                  const Real uxx = (exact_solution(x-eps,y,z) +
                                    exact_solution(x+eps,y,z) +
                                    -2.*exact_solution(x,y,z))/eps/eps;
        
                  const Real uyy = (exact_solution(x,y-eps,z) +
                                    exact_solution(x,y+eps,z) +
                                    -2.*exact_solution(x,y,z))/eps/eps;
        
                  const Real uzz = (exact_solution(x,y,z-eps) +
                                    exact_solution(x,y,z+eps) +
                                    -2.*exact_solution(x,y,z))/eps/eps;
        
                  Real fxy;
                  if(dim==1)
                  {
In 1D, compute the rhs by differentiating the exact solution twice.
                    const Real pi = libMesh::pi;
                    fxy = (0.25*pi*pi)*sin(.5*pi*x);
                  }
                  else
                  {
                    fxy = - (uxx + uyy + ((dim==2) ? 0. : uzz));
                  }
        
Add the RHS contribution
                  for (unsigned int i=0; i<phi.size(); i++)
                    Fe(i) += JxW[qp]*fxy*phi[i][qp];
                }
        
Stop logging the right-hand-side computation
              perf_log.pop ("Fe");
        
At this point the interior element integration has been completed. However, we have not yet addressed boundary conditions. For this example we will only consider simple Dirichlet boundary conditions imposed via the penalty method. This is discussed at length in example 3.
              {
        
Start logging the boundary condition computation
                perf_log.push ("BCs");
        
The following loops over the sides of the element. If the element has no neighbor on a side then that side MUST live on a boundary of the domain.
                for (unsigned int side=0; side<elem->n_sides(); side++)
                  if ((elem->neighbor(side) == NULL) ||
        	      (elem->neighbor(side)->subdomain_id() != elem->subdomain_id()))
                    {
        
The penalty value. \frac{1}{\epsilon} in the discussion above.
                      const Real penalty = 1.e10;
        
The value of the shape functions at the quadrature points.
                      const std::vector<std::vector<Real> >&  phi_face = fe_face->get_phi();
        
The Jacobian * Quadrature Weight at the quadrature points on the face.
                      const std::vector<Real>& JxW_face = fe_face->get_JxW();
        
The XYZ locations (in physical space) of the quadrature points on the face. This is where we will interpolate the boundary value function.
                      const std::vector<Point >& qface_point = fe_face->get_xyz();
        
Compute the shape function values on the element face.
                      fe_face->reinit(elem, side);
        
Loop over the face quadrature points for integration.
                      for (unsigned int qp=0; qp<qface.n_points(); qp++)
                      {
The location on the boundary of the current face quadrature point.
                        const Real xf = qface_point[qp](0);
        #if LIBMESH_DIM > 1
                        const Real yf = qface_point[qp](1);
        #else
                        const Real yf = 0.;
        #endif
        #if LIBMESH_DIM > 2
                        const Real zf = qface_point[qp](2);
        #else
                        const Real zf = 0.;
        #endif
        
        
The boundary value.
                        const Real value = exact_solution(xf, yf, zf);
        
Matrix contribution of the L2 projection.
                        for (unsigned int i=0; i<phi_face.size(); i++)
                          for (unsigned int j=0; j<phi_face.size(); j++)
                            Ke(i,j) += JxW_face[qp]*penalty*phi_face[i][qp]*phi_face[j][qp];
        
Right-hand-side contribution of the L2 projection.
                        for (unsigned int i=0; i<phi_face.size(); i++)
                          Fe(i) += JxW_face[qp]*penalty*value*phi_face[i][qp];
                      }
                    }
        
        
Stop logging the boundary condition computation
                perf_log.pop ("BCs");
              }
        
        
The element matrix and right-hand-side are now built for this element. Add them to the global matrix and right-hand-side vector. The \p PetscMatrix::add_matrix() and \p PetscVector::add_vector() members do this for us. Start logging the insertion of the local (element) matrix and vector into the global matrix and vector
              perf_log.push ("matrix insertion");
        
              if (dof_indices.size())
        	{
        	  system.matrix->add_matrix (Ke, dof_indices);
        	  system.rhs->add_vector    (Fe, dof_indices);
        	}
        
              if (dof_indices2.size())
        	{
        	  system.matrix->add_matrix (Ke, dof_indices2);
        	  system.rhs->add_vector    (Fe, dof_indices2);
        	}
        
Start logging the insertion of the local (element) matrix and vector into the global matrix and vector
              perf_log.pop ("matrix insertion");
            }
        
That's it. We don't need to do anything else to the PerfLog. When it goes out of scope (at this function return) it will print its log to the screen. Pretty easy, huh?
        }



The source file subdomains_ex2.C without comments:

 
  
  
  #include <iostream>
  #include <algorithm>
  #include <math.h>
  
  #include "libmesh/libmesh.h"
  #include "libmesh/mesh.h"
  #include "libmesh/mesh_generation.h"
  #include "libmesh/exodusII_io.h"
  #include "libmesh/gnuplot_io.h"
  #include "libmesh/linear_implicit_system.h"
  #include "libmesh/equation_systems.h"
  
  #include "libmesh/fe.h"
  
  #include "libmesh/quadrature_gauss.h"
  
  #include "libmesh/dof_map.h"
  
  #include "libmesh/sparse_matrix.h"
  #include "libmesh/numeric_vector.h"
  #include "libmesh/dense_matrix.h"
  #include "libmesh/dense_vector.h"
  
  #include "libmesh/perf_log.h"
  
  #include "libmesh/elem.h"
  
  #include "libmesh/string_to_enum.h"
  #include "libmesh/getpot.h"
  
  using namespace libMesh;
  
  
  
  void assemble_poisson(EquationSystems& es,
                        const std::string& system_name);
  
  Real exact_solution (const Real x,
                       const Real y = 0.,
                       const Real z = 0.);
  
  int main (int argc, char** argv)
  {
    LibMeshInit init (argc, argv);
  
  
    GetPot command_line (argc, argv);
  
    if (argc < 3)
      {
        if (libMesh::processor_id() == 0)
          std::cerr << "Usage:\n"
                    <<"\t " << argv[0] << " -d 2(3)" << " -n 15"
                    << std::endl;
  
        libmesh_error();
      }
  
    else
      {
        std::cout << "Running " << argv[0];
  
        for (int i=1; i<argc; i++)
          std::cout << " " << argv[i];
  
        std::cout << std::endl << std::endl;
      }
  
  
    int dim = 2;
    if ( command_line.search(1, "-d") )
      dim = command_line.next(dim);
  
    libmesh_example_assert(dim <= LIBMESH_DIM, "2D/3D support");
  
    Mesh mesh (init.comm(), dim);
  
    int ps = 15;
    if ( command_line.search(1, "-n") )
      ps = command_line.next(ps);
  
    std::string order = "SECOND";
    if ( command_line.search(2, "-Order", "-o") )
      order = command_line.next(order);
  
    std::string family = "LAGRANGE";
    if ( command_line.search(2, "-FEFamily", "-f") )
      family = command_line.next(family);
  
    if ((family == "MONOMIAL") || (family == "XYZ"))
      {
        if (libMesh::processor_id() == 0)
          std::cerr << "ex28 currently requires a C^0 (or higher) FE basis." << std::endl;
        libmesh_error();
      }
  
  
    Real halfwidth = dim > 1 ? 1. : 0.;
    Real halfheight = dim > 2 ? 1. : 0.;
  
    if ((family == "LAGRANGE") && (order == "FIRST"))
      {
        MeshTools::Generation::build_cube (mesh,
                                           ps,
  					 (dim>1) ? ps : 0,
  					 (dim>2) ? ps : 0,
                                           -1., 1.,
                                           -halfwidth, halfwidth,
                                           -halfheight, halfheight,
                                           (dim==1)    ? EDGE2 :
                                           ((dim == 2) ? QUAD4 : HEX8));
      }
  
    else
      {
        MeshTools::Generation::build_cube (mesh,
  					 ps,
  					 (dim>1) ? ps : 0,
  					 (dim>2) ? ps : 0,
                                           -1., 1.,
                                           -halfwidth, halfwidth,
                                           -halfheight, halfheight,
                                           (dim==1)    ? EDGE3 :
                                           ((dim == 2) ? QUAD9 : HEX27));
      }
  
    {
      MeshBase::element_iterator       el     = mesh.elements_begin();
      const MeshBase::element_iterator end_el = mesh.elements_end();
  
      for ( ; el != end_el; ++el)
        {
  	Elem* elem = *el;
  	const Point cent = elem->centroid();
          if (dim > 1)
            {
  	    if ((cent(0) > 0) == (cent(1) > 0))
  	      elem->subdomain_id() = 1;
            }
          else
            {
  	    if (cent(0) > 0)
  	      elem->subdomain_id() = 1;
            }
        }
    }
  
    mesh.print_info();
  
    EquationSystems equation_systems (mesh);
  
    LinearImplicitSystem& system =
      equation_systems.add_system<LinearImplicitSystem> ("Poisson");
  
  
    std::set<subdomain_id_type> active_subdomains;
  
  
    active_subdomains.clear(); active_subdomains.insert(0);
    system.add_variable("u",
                        Utility::string_to_enum<Order>   (order),
                        Utility::string_to_enum<FEFamily>(family),
  		      &active_subdomains);
  
    active_subdomains.clear(); active_subdomains.insert(1);
    system.add_variable("v",
                        Utility::string_to_enum<Order>   (order),
                        Utility::string_to_enum<FEFamily>(family),
  		      &active_subdomains);
  
    system.attach_assemble_function (assemble_poisson);
  
    equation_systems.init();
  
    equation_systems.print_info();
    mesh.print_info();
  
    equation_systems.get_system("Poisson").solve();
  
    if(dim == 1)
    {
      GnuPlotIO plot(mesh,"Subdomains Example 2, 1D",GnuPlotIO::GRID_ON);
      plot.write_equation_systems("gnuplot_script",equation_systems);
    }
    else
    {
  #ifdef LIBMESH_HAVE_EXODUS_API
      ExodusII_IO (mesh).write_equation_systems ((dim == 3) ?
        "out_3.e" : "out_2.e",equation_systems);
  #endif // #ifdef LIBMESH_HAVE_EXODUS_API
    }
  
    return 0;
  }
  
  
  
  
  void assemble_poisson(EquationSystems& es,
                        const std::string& system_name)
  {
    libmesh_assert_equal_to (system_name, "Poisson");
  
    PerfLog perf_log ("Matrix Assembly");
  
    const MeshBase& mesh = es.get_mesh();
  
    const unsigned int dim = mesh.mesh_dimension();
  
    LinearImplicitSystem& system = es.get_system<LinearImplicitSystem>("Poisson");
  
    const DofMap& dof_map = system.get_dof_map();
  
    FEType fe_type = dof_map.variable_type(0);
  
    AutoPtr<FEBase> fe (FEBase::build(dim, fe_type));
  
    QGauss qrule (dim, FIFTH);
  
    fe->attach_quadrature_rule (&qrule);
  
    AutoPtr<FEBase> fe_face (FEBase::build(dim, fe_type));
  
    QGauss qface(dim-1, FIFTH);
  
    fe_face->attach_quadrature_rule (&qface);
  
    const std::vector<Real>& JxW = fe->get_JxW();
  
    const std::vector<Point>& q_point = fe->get_xyz();
  
    const std::vector<std::vector<Real> >& phi = fe->get_phi();
  
    const std::vector<std::vector<RealGradient> >& dphi = fe->get_dphi();
  
    DenseMatrix<Number> Ke;
    DenseVector<Number> Fe;
  
    std::vector<dof_id_type> dof_indices, dof_indices2;
  
    MeshBase::const_element_iterator       el     = mesh.local_elements_begin();
    const MeshBase::const_element_iterator end_el = mesh.local_elements_end();
  
    for ( ; el != end_el; ++el)
      {
        perf_log.push("elem init");
  
        const Elem* elem = *el;
  
        dof_map.dof_indices (elem, dof_indices,0);
        dof_map.dof_indices (elem, dof_indices2,1);
  
  
        fe->reinit (elem);
  
        Ke.resize (std::max(dof_indices.size(), dof_indices2.size()),
  		 std::max(dof_indices.size(), dof_indices2.size()));
  
        Fe.resize (std::max(dof_indices.size(), dof_indices2.size()));
  
        perf_log.pop("elem init");
  
        perf_log.push ("Ke");
  
        for (unsigned int qp=0; qp<qrule.n_points(); qp++)
          for (unsigned int i=0; i<phi.size(); i++)
            for (unsigned int j=0; j<phi.size(); j++)
              Ke(i,j) += JxW[qp]*(dphi[i][qp]*dphi[j][qp]);
  
  
        perf_log.pop ("Ke");
  
        perf_log.push ("Fe");
  
        for (unsigned int qp=0; qp<qrule.n_points(); qp++)
          {
            const Real x = q_point[qp](0);
  #if LIBMESH_DIM > 1
            const Real y = q_point[qp](1);
  #else
            const Real y = 0;
  #endif
  #if LIBMESH_DIM > 2
            const Real z = q_point[qp](2);
  #else
            const Real z = 0;
  #endif
            const Real eps = 1.e-3;
  
            const Real uxx = (exact_solution(x-eps,y,z) +
                              exact_solution(x+eps,y,z) +
                              -2.*exact_solution(x,y,z))/eps/eps;
  
            const Real uyy = (exact_solution(x,y-eps,z) +
                              exact_solution(x,y+eps,z) +
                              -2.*exact_solution(x,y,z))/eps/eps;
  
            const Real uzz = (exact_solution(x,y,z-eps) +
                              exact_solution(x,y,z+eps) +
                              -2.*exact_solution(x,y,z))/eps/eps;
  
            Real fxy;
            if(dim==1)
            {
              const Real pi = libMesh::pi;
              fxy = (0.25*pi*pi)*sin(.5*pi*x);
            }
            else
            {
              fxy = - (uxx + uyy + ((dim==2) ? 0. : uzz));
            }
  
            for (unsigned int i=0; i<phi.size(); i++)
              Fe(i) += JxW[qp]*fxy*phi[i][qp];
          }
  
        perf_log.pop ("Fe");
  
        {
  
          perf_log.push ("BCs");
  
          for (unsigned int side=0; side<elem->n_sides(); side++)
            if ((elem->neighbor(side) == NULL) ||
  	      (elem->neighbor(side)->subdomain_id() != elem->subdomain_id()))
              {
  
                const Real penalty = 1.e10;
  
                const std::vector<std::vector<Real> >&  phi_face = fe_face->get_phi();
  
                const std::vector<Real>& JxW_face = fe_face->get_JxW();
  
                const std::vector<Point >& qface_point = fe_face->get_xyz();
  
                fe_face->reinit(elem, side);
  
                for (unsigned int qp=0; qp<qface.n_points(); qp++)
                {
                  const Real xf = qface_point[qp](0);
  #if LIBMESH_DIM > 1
                  const Real yf = qface_point[qp](1);
  #else
                  const Real yf = 0.;
  #endif
  #if LIBMESH_DIM > 2
                  const Real zf = qface_point[qp](2);
  #else
                  const Real zf = 0.;
  #endif
  
  
                  const Real value = exact_solution(xf, yf, zf);
  
                  for (unsigned int i=0; i<phi_face.size(); i++)
                    for (unsigned int j=0; j<phi_face.size(); j++)
                      Ke(i,j) += JxW_face[qp]*penalty*phi_face[i][qp]*phi_face[j][qp];
  
                  for (unsigned int i=0; i<phi_face.size(); i++)
                    Fe(i) += JxW_face[qp]*penalty*value*phi_face[i][qp];
                }
              }
  
  
          perf_log.pop ("BCs");
        }
  
  
        perf_log.push ("matrix insertion");
  
        if (dof_indices.size())
  	{
  	  system.matrix->add_matrix (Ke, dof_indices);
  	  system.rhs->add_vector    (Fe, dof_indices);
  	}
  
        if (dof_indices2.size())
  	{
  	  system.matrix->add_matrix (Ke, dof_indices2);
  	  system.rhs->add_vector    (Fe, dof_indices2);
  	}
  
        perf_log.pop ("matrix insertion");
      }
  
  }



The console output of the program:

make[4]: Entering directory `/net/spark/workspace/roystgnr/libmesh/git/devel/examples/subdomains/subdomains_ex2'
***************************************************************
* Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 1 -n 20 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
 
Running /net/spark/workspace/roystgnr/libmesh/git/devel/examples/subdomains/subdomains_ex2/.libs/lt-example-devel -d 1 -n 20 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc

 Mesh Information:
  mesh_dimension()=1
  spatial_dimension()=3
  n_nodes()=41
    n_local_nodes()=11
  n_elem()=20
    n_local_elem()=5
    n_active_elem()=20
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0

 EquationSystems
  n_systems()=1
   System #0, "Poisson"
    Type "LinearImplicit"
    Variables="u" "v" 
    Finite Element Types="LAGRANGE", "JACOBI_20_00" "LAGRANGE", "JACOBI_20_00" 
    Infinite Element Mapping="CARTESIAN" "CARTESIAN" 
    Approximation Orders="SECOND", "THIRD" "SECOND", "THIRD" 
    n_dofs()=42
    n_local_dofs()=11
    n_constrained_dofs()=0
    n_local_constrained_dofs()=0
    n_vectors()=1
    n_matrices()=1
    DofMap Sparsity
      Average  On-Processor Bandwidth <= 3.61905
      Average Off-Processor Bandwidth <= 0.285714
      Maximum  On-Processor Bandwidth <= 5
      Maximum Off-Processor Bandwidth <= 2
    DofMap Constraints
      Number of DoF Constraints = 0
      Number of Node Constraints = 0

 Mesh Information:
  mesh_dimension()=1
  spatial_dimension()=3
  n_nodes()=41
    n_local_nodes()=11
  n_elem()=20
    n_local_elem()=5
    n_active_elem()=20
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0


 -------------------------------------------------------------------------------------------------------------------
| Processor id:   0                                                                                                 |
| Num Processors: 4                                                                                                 |
| Time:           Fri Apr 19 11:52:46 2013                                                                          |
| OS:             Linux                                                                                             |
| HostName:       spark.ices.utexas.edu                                                                             |
| OS Release:     2.6.32-279.22.1.el6.x86_64                                                                        |
| OS Version:     #1 SMP Tue Feb 5 14:33:39 CST 2013                                                                |
| Machine:        x86_64                                                                                            |
| Username:       roystgnr                                                                                          |
| Configuration:  ../configure  '--enable-everything'                                                               |
|  'METHODS=devel'                                                                                                  |
|  '--prefix=/h2/roystgnr/libmesh-test'                                                                             |
|  'CXX=distcc /usr/bin/g++'                                                                                        |
|  'CC=distcc /usr/bin/gcc'                                                                                         |
|  'FC=distcc /usr/bin/gfortran'                                                                                    |
|  'F77=distcc /usr/bin/gfortran'                                                                                   |
|  'PETSC_DIR=/opt/apps/ossw/libraries/petsc/petsc-3.3-p2'                                                          |
|  'PETSC_ARCH=gcc-system-mkl-gf-10.3.12.361-mpich2-1.4.1p1-cxx-opt'                                                |
|  'SLEPC_DIR=/opt/apps/ossw/libraries/slepc/slepc-3.3-p2-petsc-3.3-p2-cxx-opt'                                     |
|  'TRILINOS_DIR=/opt/apps/ossw/libraries/trilinos/trilinos-10.12.2/sl6/gcc-system/mpich2-1.4.1p1/mkl-gf-10.3.12.361'|
|  'VTK_DIR=/opt/apps/ossw/libraries/vtk/vtk-5.10.0/sl6/gcc-system'                                                 |
|  'HDF5_DIR=/opt/apps/ossw/libraries/hdf5/hdf5-1.8.9/sl6/gcc-system'                                               |
 -------------------------------------------------------------------------------------------------------------------
 -----------------------------------------------------------------------------------------------------------
| Matrix Assembly Performance: Alive time=0.000735, Active time=0.000413                                    |
 -----------------------------------------------------------------------------------------------------------
| Event                         nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                         w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|-----------------------------------------------------------------------------------------------------------|
|                                                                                                           |
| BCs                           5         0.0001      0.000017    0.0001      0.000017    20.34    20.34    |
| Fe                            5         0.0001      0.000011    0.0001      0.000011    12.83    12.83    |
| Ke                            5         0.0000      0.000001    0.0000      0.000001    1.69     1.69     |
| elem init                     5         0.0002      0.000045    0.0002      0.000045    54.96    54.96    |
| matrix insertion              5         0.0000      0.000008    0.0000      0.000008    10.17    10.17    |
 -----------------------------------------------------------------------------------------------------------
| Totals:                       25        0.0004                                          100.00            |
 -----------------------------------------------------------------------------------------------------------

 ----------------------------------------------------------------------------------------------------------------
| libMesh Performance: Alive time=0.20397, Active time=0.026593                                                  |
 ----------------------------------------------------------------------------------------------------------------
| Event                              nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                              w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|----------------------------------------------------------------------------------------------------------------|
|                                                                                                                |
|                                                                                                                |
| DofMap                                                                                                         |
|   add_neighbors_to_send_list()     1         0.0001      0.000068    0.0001      0.000080    0.26     0.30     |
|   build_sparsity()                 1         0.0002      0.000216    0.0005      0.000519    0.81     1.95     |
|   create_dof_constraints()         1         0.0000      0.000001    0.0000      0.000001    0.00     0.00     |
|   distribute_dofs()                1         0.0003      0.000263    0.0008      0.000826    0.99     3.11     |
|   dof_indices()                    22        0.0001      0.000005    0.0001      0.000005    0.37     0.37     |
|   prepare_send_list()              1         0.0000      0.000002    0.0000      0.000002    0.01     0.01     |
|   reinit()                         1         0.0002      0.000202    0.0002      0.000202    0.76     0.76     |
|                                                                                                                |
| EquationSystems                                                                                                |
|   build_solution_vector()          1         0.0002      0.000178    0.0006      0.000562    0.67     2.11     |
|                                                                                                                |
| FE                                                                                                             |
|   compute_shape_functions()        6         0.0000      0.000005    0.0000      0.000005    0.12     0.12     |
|   init_shape_functions()           2         0.0000      0.000016    0.0000      0.000016    0.12     0.12     |
|                                                                                                                |
| FEMap                                                                                                          |
|   compute_affine_map()             6         0.0000      0.000005    0.0000      0.000005    0.11     0.11     |
|   compute_face_map()               1         0.0000      0.000007    0.0000      0.000007    0.03     0.03     |
|   init_face_shape_functions()      1         0.0000      0.000007    0.0000      0.000007    0.03     0.03     |
|   init_reference_to_physical_map() 2         0.0000      0.000015    0.0000      0.000015    0.11     0.11     |
|                                                                                                                |
| GnuPlotIO                                                                                                      |
|   write_nodal_data()               1         0.0008      0.000771    0.0008      0.000771    2.90     2.90     |
|                                                                                                                |
| Mesh                                                                                                           |
|   find_neighbors()                 1         0.0002      0.000247    0.0003      0.000327    0.93     1.23     |
|   renumber_nodes_and_elem()        2         0.0004      0.000204    0.0004      0.000204    1.53     1.53     |
|                                                                                                                |
| MeshCommunication                                                                                              |
|   compute_hilbert_indices()        2         0.0003      0.000149    0.0003      0.000149    1.12     1.12     |
|   find_global_indices()            2         0.0002      0.000077    0.0016      0.000815    0.58     6.13     |
|   parallel_sort()                  2         0.0007      0.000354    0.0008      0.000415    2.67     3.12     |
|                                                                                                                |
| MeshOutput                                                                                                     |
|   write_equation_systems()         1         0.0001      0.000106    0.0023      0.002325    0.40     8.74     |
|                                                                                                                |
| MeshTools::Generation                                                                                          |
|   build_cube()                     1         0.0001      0.000086    0.0001      0.000086    0.32     0.32     |
|                                                                                                                |
| MetisPartitioner                                                                                               |
|   partition()                      1         0.0008      0.000773    0.0013      0.001261    2.91     4.74     |
|                                                                                                                |
| Parallel                                                                                                       |
|   allgather()                      11        0.0002      0.000017    0.0002      0.000021    0.70     0.87     |
|   max(bool)                        1         0.0000      0.000006    0.0000      0.000006    0.02     0.02     |
|   max(scalar)                      115       0.0009      0.000008    0.0009      0.000008    3.29     3.29     |
|   max(vector)                      26        0.0002      0.000009    0.0007      0.000027    0.83     2.61     |
|   min(bool)                        133       0.0008      0.000006    0.0008      0.000006    2.84     2.84     |
|   min(scalar)                      109       0.0012      0.000011    0.0012      0.000011    4.61     4.61     |
|   min(vector)                      26        0.0003      0.000012    0.0008      0.000031    1.15     3.05     |
|   probe()                          36        0.0002      0.000004    0.0002      0.000004    0.57     0.57     |
|   receive()                        36        0.0001      0.000003    0.0003      0.000008    0.44     1.03     |
|   send()                           36        0.0001      0.000002    0.0001      0.000002    0.32     0.32     |
|   send_receive()                   40        0.0002      0.000006    0.0007      0.000017    0.86     2.48     |
|   sum()                            22        0.0002      0.000008    0.0004      0.000017    0.70     1.37     |
|                                                                                                                |
| Parallel::Request                                                                                              |
|   wait()                           36        0.0001      0.000002    0.0001      0.000002    0.22     0.22     |
|                                                                                                                |
| Partitioner                                                                                                    |
|   set_node_processor_ids()         1         0.0001      0.000121    0.0003      0.000290    0.46     1.09     |
|   set_parent_processor_ids()       1         0.0000      0.000022    0.0000      0.000022    0.08     0.08     |
|                                                                                                                |
| PetscLinearSolver                                                                                              |
|   solve()                          1         0.0166      0.016562    0.0166      0.016562    62.28    62.28    |
|                                                                                                                |
| System                                                                                                         |
|   assemble()                       1         0.0008      0.000772    0.0009      0.000942    2.90     3.54     |
 ----------------------------------------------------------------------------------------------------------------
| Totals:                            691       0.0266                                          100.00            |
 ----------------------------------------------------------------------------------------------------------------

 
***************************************************************
* Done Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 1 -n 20 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
***************************************************************
* Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 2 -n 15 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
 
Running /net/spark/workspace/roystgnr/libmesh/git/devel/examples/subdomains/subdomains_ex2/.libs/lt-example-devel -d 2 -n 15 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc

 Mesh Information:
  mesh_dimension()=2
  spatial_dimension()=3
  n_nodes()=961
    n_local_nodes()=255
  n_elem()=225
    n_local_elem()=56
    n_active_elem()=225
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0

 EquationSystems
  n_systems()=1
   System #0, "Poisson"
    Type "LinearImplicit"
    Variables="u" "v" 
    Finite Element Types="LAGRANGE", "JACOBI_20_00" "LAGRANGE", "JACOBI_20_00" 
    Infinite Element Mapping="CARTESIAN" "CARTESIAN" 
    Approximation Orders="SECOND", "THIRD" "SECOND", "THIRD" 
    n_dofs()=1022
    n_local_dofs()=270
    n_constrained_dofs()=0
    n_local_constrained_dofs()=0
    n_vectors()=1
    n_matrices()=1
    DofMap Sparsity
      Average  On-Processor Bandwidth <= 13.7319
      Average Off-Processor Bandwidth <= 0.951076
      Maximum  On-Processor Bandwidth <= 25
      Maximum Off-Processor Bandwidth <= 16
    DofMap Constraints
      Number of DoF Constraints = 0
      Number of Node Constraints = 0

 Mesh Information:
  mesh_dimension()=2
  spatial_dimension()=3
  n_nodes()=961
    n_local_nodes()=255
  n_elem()=225
    n_local_elem()=56
    n_active_elem()=225
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0


 -------------------------------------------------------------------------------------------------------------------
| Processor id:   0                                                                                                 |
| Num Processors: 4                                                                                                 |
| Time:           Fri Apr 19 11:52:47 2013                                                                          |
| OS:             Linux                                                                                             |
| HostName:       spark.ices.utexas.edu                                                                             |
| OS Release:     2.6.32-279.22.1.el6.x86_64                                                                        |
| OS Version:     #1 SMP Tue Feb 5 14:33:39 CST 2013                                                                |
| Machine:        x86_64                                                                                            |
| Username:       roystgnr                                                                                          |
| Configuration:  ../configure  '--enable-everything'                                                               |
|  'METHODS=devel'                                                                                                  |
|  '--prefix=/h2/roystgnr/libmesh-test'                                                                             |
|  'CXX=distcc /usr/bin/g++'                                                                                        |
|  'CC=distcc /usr/bin/gcc'                                                                                         |
|  'FC=distcc /usr/bin/gfortran'                                                                                    |
|  'F77=distcc /usr/bin/gfortran'                                                                                   |
|  'PETSC_DIR=/opt/apps/ossw/libraries/petsc/petsc-3.3-p2'                                                          |
|  'PETSC_ARCH=gcc-system-mkl-gf-10.3.12.361-mpich2-1.4.1p1-cxx-opt'                                                |
|  'SLEPC_DIR=/opt/apps/ossw/libraries/slepc/slepc-3.3-p2-petsc-3.3-p2-cxx-opt'                                     |
|  'TRILINOS_DIR=/opt/apps/ossw/libraries/trilinos/trilinos-10.12.2/sl6/gcc-system/mpich2-1.4.1p1/mkl-gf-10.3.12.361'|
|  'VTK_DIR=/opt/apps/ossw/libraries/vtk/vtk-5.10.0/sl6/gcc-system'                                                 |
|  'HDF5_DIR=/opt/apps/ossw/libraries/hdf5/hdf5-1.8.9/sl6/gcc-system'                                               |
 -------------------------------------------------------------------------------------------------------------------
 -----------------------------------------------------------------------------------------------------------
| Matrix Assembly Performance: Alive time=0.007019, Active time=0.00651                                     |
 -----------------------------------------------------------------------------------------------------------
| Event                         nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                         w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|-----------------------------------------------------------------------------------------------------------|
|                                                                                                           |
| BCs                           56        0.0021      0.000037    0.0021      0.000037    32.10    32.10    |
| Fe                            56        0.0005      0.000009    0.0005      0.000009    7.99     7.99     |
| Ke                            56        0.0013      0.000023    0.0013      0.000023    19.55    19.55    |
| elem init                     56        0.0022      0.000040    0.0022      0.000040    34.01    34.01    |
| matrix insertion              56        0.0004      0.000007    0.0004      0.000007    6.34     6.34     |
 -----------------------------------------------------------------------------------------------------------
| Totals:                       280       0.0065                                          100.00            |
 -----------------------------------------------------------------------------------------------------------

 ----------------------------------------------------------------------------------------------------------------
| libMesh Performance: Alive time=0.16675, Active time=0.153412                                                  |
 ----------------------------------------------------------------------------------------------------------------
| Event                              nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                              w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|----------------------------------------------------------------------------------------------------------------|
|                                                                                                                |
|                                                                                                                |
| DofMap                                                                                                         |
|   add_neighbors_to_send_list()     1         0.0009      0.000882    0.0013      0.001277    0.57     0.83     |
|   build_sparsity()                 1         0.0004      0.000443    0.0016      0.001603    0.29     1.04     |
|   create_dof_constraints()         1         0.0001      0.000125    0.0001      0.000125    0.08     0.08     |
|   distribute_dofs()                1         0.0022      0.002191    0.0052      0.005157    1.43     3.36     |
|   dof_indices()                    255       0.0025      0.000010    0.0025      0.000010    1.64     1.64     |
|   prepare_send_list()              1         0.0000      0.000010    0.0000      0.000010    0.01     0.01     |
|   reinit()                         1         0.0023      0.002334    0.0023      0.002334    1.52     1.52     |
|                                                                                                                |
| EquationSystems                                                                                                |
|   build_solution_vector()          1         0.0004      0.000363    0.0066      0.006631    0.24     4.32     |
|                                                                                                                |
| ExodusII_IO                                                                                                    |
|   write_nodal_data()               1         0.0888      0.088759    0.0888      0.088759    57.86    57.86    |
|                                                                                                                |
| FE                                                                                                             |
|   compute_shape_functions()        85        0.0007      0.000008    0.0007      0.000008    0.46     0.46     |
|   init_shape_functions()           30        0.0001      0.000003    0.0001      0.000003    0.06     0.06     |
|   inverse_map()                    87        0.0006      0.000006    0.0006      0.000006    0.36     0.36     |
|                                                                                                                |
| FEMap                                                                                                          |
|   compute_affine_map()             85        0.0005      0.000005    0.0005      0.000005    0.30     0.30     |
|   compute_face_map()               29        0.0003      0.000011    0.0009      0.000031    0.21     0.58     |
|   init_face_shape_functions()      1         0.0000      0.000005    0.0000      0.000005    0.00     0.00     |
|   init_reference_to_physical_map() 30        0.0004      0.000013    0.0004      0.000013    0.25     0.25     |
|                                                                                                                |
| Mesh                                                                                                           |
|   find_neighbors()                 1         0.0009      0.000865    0.0027      0.002688    0.56     1.75     |
|   renumber_nodes_and_elem()        2         0.0002      0.000085    0.0002      0.000085    0.11     0.11     |
|                                                                                                                |
| MeshCommunication                                                                                              |
|   compute_hilbert_indices()        2         0.0019      0.000949    0.0019      0.000949    1.24     1.24     |
|   find_global_indices()            2         0.0004      0.000178    0.0032      0.001615    0.23     2.11     |
|   parallel_sort()                  2         0.0004      0.000207    0.0007      0.000327    0.27     0.43     |
|                                                                                                                |
| MeshOutput                                                                                                     |
|   write_equation_systems()         1         0.0001      0.000060    0.0966      0.096624    0.04     62.98    |
|                                                                                                                |
| MeshTools::Generation                                                                                          |
|   build_cube()                     1         0.0005      0.000532    0.0005      0.000532    0.35     0.35     |
|                                                                                                                |
| MetisPartitioner                                                                                               |
|   partition()                      1         0.0026      0.002607    0.0041      0.004081    1.70     2.66     |
|                                                                                                                |
| Parallel                                                                                                       |
|   allgather()                      11        0.0003      0.000030    0.0004      0.000034    0.22     0.24     |
|   max(bool)                        1         0.0000      0.000007    0.0000      0.000007    0.00     0.00     |
|   max(scalar)                      115       0.0040      0.000035    0.0040      0.000035    2.60     2.60     |
|   max(vector)                      26        0.0007      0.000027    0.0026      0.000099    0.45     1.68     |
|   min(bool)                        133       0.0035      0.000026    0.0035      0.000026    2.27     2.27     |
|   min(scalar)                      109       0.0068      0.000063    0.0068      0.000063    4.45     4.45     |
|   min(vector)                      26        0.0008      0.000029    0.0027      0.000102    0.49     1.73     |
|   probe()                          36        0.0002      0.000004    0.0002      0.000004    0.10     0.10     |
|   receive()                        36        0.0001      0.000004    0.0003      0.000008    0.09     0.19     |
|   send()                           36        0.0001      0.000003    0.0001      0.000003    0.07     0.07     |
|   send_receive()                   40        0.0002      0.000006    0.0007      0.000017    0.15     0.45     |
|   sum()                            22        0.0009      0.000040    0.0033      0.000151    0.58     2.16     |
|                                                                                                                |
| Parallel::Request                                                                                              |
|   wait()                           36        0.0001      0.000002    0.0001      0.000002    0.04     0.04     |
|                                                                                                                |
| Partitioner                                                                                                    |
|   set_node_processor_ids()         1         0.0005      0.000482    0.0007      0.000702    0.31     0.46     |
|   set_parent_processor_ids()       1         0.0001      0.000078    0.0001      0.000078    0.05     0.05     |
|                                                                                                                |
| PetscLinearSolver                                                                                              |
|   solve()                          1         0.0244      0.024357    0.0244      0.024357    15.88    15.88    |
|                                                                                                                |
| System                                                                                                         |
|   assemble()                       1         0.0038      0.003783    0.0071      0.007140    2.47     4.65     |
 ----------------------------------------------------------------------------------------------------------------
| Totals:                            1253      0.1534                                          100.00            |
 ----------------------------------------------------------------------------------------------------------------

 
***************************************************************
* Done Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 2 -n 15 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
***************************************************************
* Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 3 -n 6 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
 
Running /net/spark/workspace/roystgnr/libmesh/git/devel/examples/subdomains/subdomains_ex2/.libs/lt-example-devel -d 3 -n 6 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc

 Mesh Information:
  mesh_dimension()=3
  spatial_dimension()=3
  n_nodes()=2197
    n_local_nodes()=655
  n_elem()=216
    n_local_elem()=54
    n_active_elem()=216
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0

 EquationSystems
  n_systems()=1
   System #0, "Poisson"
    Type "LinearImplicit"
    Variables="u" "v" 
    Finite Element Types="LAGRANGE", "JACOBI_20_00" "LAGRANGE", "JACOBI_20_00" 
    Infinite Element Mapping="CARTESIAN" "CARTESIAN" 
    Approximation Orders="SECOND", "THIRD" "SECOND", "THIRD" 
    n_dofs()=2522
    n_local_dofs()=720
    n_constrained_dofs()=0
    n_local_constrained_dofs()=0
    n_vectors()=1
    n_matrices()=1
    DofMap Sparsity
      Average  On-Processor Bandwidth <= 41.4417
      Average Off-Processor Bandwidth <= 9.30849
      Maximum  On-Processor Bandwidth <= 136
      Maximum Off-Processor Bandwidth <= 92
    DofMap Constraints
      Number of DoF Constraints = 0
      Number of Node Constraints = 0

 Mesh Information:
  mesh_dimension()=3
  spatial_dimension()=3
  n_nodes()=2197
    n_local_nodes()=655
  n_elem()=216
    n_local_elem()=54
    n_active_elem()=216
  n_subdomains()=2
  n_partitions()=4
  n_processors()=4
  n_threads()=1
  processor_id()=0


 -------------------------------------------------------------------------------------------------------------------
| Processor id:   0                                                                                                 |
| Num Processors: 4                                                                                                 |
| Time:           Fri Apr 19 11:52:47 2013                                                                          |
| OS:             Linux                                                                                             |
| HostName:       spark.ices.utexas.edu                                                                             |
| OS Release:     2.6.32-279.22.1.el6.x86_64                                                                        |
| OS Version:     #1 SMP Tue Feb 5 14:33:39 CST 2013                                                                |
| Machine:        x86_64                                                                                            |
| Username:       roystgnr                                                                                          |
| Configuration:  ../configure  '--enable-everything'                                                               |
|  'METHODS=devel'                                                                                                  |
|  '--prefix=/h2/roystgnr/libmesh-test'                                                                             |
|  'CXX=distcc /usr/bin/g++'                                                                                        |
|  'CC=distcc /usr/bin/gcc'                                                                                         |
|  'FC=distcc /usr/bin/gfortran'                                                                                    |
|  'F77=distcc /usr/bin/gfortran'                                                                                   |
|  'PETSC_DIR=/opt/apps/ossw/libraries/petsc/petsc-3.3-p2'                                                          |
|  'PETSC_ARCH=gcc-system-mkl-gf-10.3.12.361-mpich2-1.4.1p1-cxx-opt'                                                |
|  'SLEPC_DIR=/opt/apps/ossw/libraries/slepc/slepc-3.3-p2-petsc-3.3-p2-cxx-opt'                                     |
|  'TRILINOS_DIR=/opt/apps/ossw/libraries/trilinos/trilinos-10.12.2/sl6/gcc-system/mpich2-1.4.1p1/mkl-gf-10.3.12.361'|
|  'VTK_DIR=/opt/apps/ossw/libraries/vtk/vtk-5.10.0/sl6/gcc-system'                                                 |
|  'HDF5_DIR=/opt/apps/ossw/libraries/hdf5/hdf5-1.8.9/sl6/gcc-system'                                               |
 -------------------------------------------------------------------------------------------------------------------
 -----------------------------------------------------------------------------------------------------------
| Matrix Assembly Performance: Alive time=0.081199, Active time=0.080613                                    |
 -----------------------------------------------------------------------------------------------------------
| Event                         nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                         w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|-----------------------------------------------------------------------------------------------------------|
|                                                                                                           |
| BCs                           54        0.0316      0.000586    0.0316      0.000586    39.24    39.24    |
| Fe                            54        0.0017      0.000032    0.0017      0.000032    2.15     2.15     |
| Ke                            54        0.0335      0.000621    0.0335      0.000621    41.58    41.58    |
| elem init                     54        0.0112      0.000208    0.0112      0.000208    13.92    13.92    |
| matrix insertion              54        0.0025      0.000046    0.0025      0.000046    3.11     3.11     |
 -----------------------------------------------------------------------------------------------------------
| Totals:                       270       0.0806                                          100.00            |
 -----------------------------------------------------------------------------------------------------------

 ----------------------------------------------------------------------------------------------------------------
| libMesh Performance: Alive time=0.324138, Active time=0.285096                                                 |
 ----------------------------------------------------------------------------------------------------------------
| Event                              nCalls    Total Time  Avg Time    Total Time  Avg Time    % of Active Time  |
|                                              w/o Sub     w/o Sub     With Sub    With Sub    w/o S    With S   |
|----------------------------------------------------------------------------------------------------------------|
|                                                                                                                |
|                                                                                                                |
| DofMap                                                                                                         |
|   add_neighbors_to_send_list()     1         0.0026      0.002611    0.0060      0.005982    0.92     2.10     |
|   build_sparsity()                 1         0.0017      0.001747    0.0045      0.004547    0.61     1.59     |
|   create_dof_constraints()         1         0.0002      0.000151    0.0002      0.000151    0.05     0.05     |
|   distribute_dofs()                1         0.0047      0.004740    0.0104      0.010450    1.66     3.67     |
|   dof_indices()                    305       0.0093      0.000031    0.0093      0.000031    3.27     3.27     |
|   prepare_send_list()              1         0.0001      0.000086    0.0001      0.000086    0.03     0.03     |
|   reinit()                         1         0.0053      0.005301    0.0053      0.005301    1.86     1.86     |
|                                                                                                                |
| EquationSystems                                                                                                |
|   build_solution_vector()          1         0.0004      0.000436    0.0036      0.003636    0.15     1.28     |
|                                                                                                                |
| ExodusII_IO                                                                                                    |
|   write_nodal_data()               1         0.0596      0.059639    0.0596      0.059639    20.92    20.92    |
|                                                                                                                |
| FE                                                                                                             |
|   compute_shape_functions()        135       0.0078      0.000058    0.0078      0.000058    2.75     2.75     |
|   init_shape_functions()           82        0.0004      0.000005    0.0004      0.000005    0.16     0.16     |
|                                                                                                                |
| FEMap                                                                                                          |
|   compute_affine_map()             135       0.0031      0.000023    0.0031      0.000023    1.08     1.08     |
|   compute_face_map()               81        0.0012      0.000015    0.0012      0.000015    0.42     0.42     |
|   init_face_shape_functions()      1         0.0000      0.000042    0.0000      0.000042    0.01     0.01     |
|   init_reference_to_physical_map() 82        0.0166      0.000202    0.0166      0.000202    5.81     5.81     |
|                                                                                                                |
| Mesh                                                                                                           |
|   find_neighbors()                 1         0.0105      0.010539    0.0106      0.010650    3.70     3.74     |
|   renumber_nodes_and_elem()        2         0.0004      0.000189    0.0004      0.000189    0.13     0.13     |
|                                                                                                                |
| MeshCommunication                                                                                              |
|   compute_hilbert_indices()        2         0.0019      0.000962    0.0019      0.000962    0.67     0.67     |
|   find_global_indices()            2         0.0003      0.000167    0.0030      0.001492    0.12     1.05     |
|   parallel_sort()                  2         0.0004      0.000185    0.0005      0.000226    0.13     0.16     |
|                                                                                                                |
| MeshOutput                                                                                                     |
|   write_equation_systems()         1         0.0001      0.000063    0.0634      0.063407    0.02     22.24    |
|                                                                                                                |
| MeshTools::Generation                                                                                          |
|   build_cube()                     1         0.0013      0.001258    0.0013      0.001258    0.44     0.44     |
|                                                                                                                |
| MetisPartitioner                                                                                               |
|   partition()                      1         0.0040      0.004050    0.0055      0.005479    1.42     1.92     |
|                                                                                                                |
| Parallel                                                                                                       |
|   allgather()                      11        0.0001      0.000010    0.0001      0.000013    0.04     0.05     |
|   max(bool)                        1         0.0000      0.000003    0.0000      0.000003    0.00     0.00     |
|   max(scalar)                      115       0.0005      0.000004    0.0005      0.000004    0.17     0.17     |
|   max(vector)                      26        0.0002      0.000006    0.0005      0.000018    0.06     0.17     |
|   min(bool)                        133       0.0005      0.000003    0.0005      0.000003    0.16     0.16     |
|   min(scalar)                      109       0.0031      0.000028    0.0031      0.000028    1.07     1.07     |
|   min(vector)                      26        0.0002      0.000009    0.0005      0.000019    0.08     0.18     |
|   probe()                          36        0.0002      0.000007    0.0002      0.000007    0.09     0.09     |
|   receive()                        36        0.0002      0.000004    0.0004      0.000011    0.05     0.14     |
|   send()                           36        0.0001      0.000003    0.0001      0.000003    0.04     0.04     |
|   send_receive()                   40        0.0003      0.000007    0.0009      0.000021    0.10     0.30     |
|   sum()                            22        0.0004      0.000017    0.0013      0.000057    0.13     0.44     |
|                                                                                                                |
| Parallel::Request                                                                                              |
|   wait()                           36        0.0001      0.000002    0.0001      0.000002    0.02     0.02     |
|                                                                                                                |
| Partitioner                                                                                                    |
|   set_node_processor_ids()         1         0.0009      0.000883    0.0010      0.001012    0.31     0.35     |
|   set_parent_processor_ids()       1         0.0001      0.000087    0.0001      0.000087    0.03     0.03     |
|                                                                                                                |
| PetscLinearSolver                                                                                              |
|   solve()                          1         0.0963      0.096274    0.0963      0.096274    33.77    33.77    |
|                                                                                                                |
| System                                                                                                         |
|   assemble()                       1         0.0500      0.050041    0.0813      0.081335    17.55    28.53    |
 ----------------------------------------------------------------------------------------------------------------
| Totals:                            1472      0.2851                                          100.00            |
 ----------------------------------------------------------------------------------------------------------------

 
***************************************************************
* Done Running Example subdomains_ex2:
*  mpirun -np 4 example-devel -d 3 -n 6 -pc_type bjacobi -sub_pc_type ilu -sub_pc_factor_levels 4 -sub_pc_factor_zeropivot 0 -ksp_right_pc
***************************************************************
make[4]: Leaving directory `/net/spark/workspace/roystgnr/libmesh/git/devel/examples/subdomains/subdomains_ex2'

Site Created By: libMesh Developers
Last modified: April 23 2013 04:19:30 UTC

Hosted By:
SourceForge.net Logo